Jul 18 01:46:17.580257 ip-10-0-142-56 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 18 01:46:17.580272 ip-10-0-142-56 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 18 01:46:17.580282 ip-10-0-142-56 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 18 01:46:17.580536 ip-10-0-142-56 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 18 01:46:27.618517 ip-10-0-142-56 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 18 01:46:27.618539 ip-10-0-142-56 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b8979665510e46168a2632de319eea96 --
Jul 18 01:48:26.666348 ip-10-0-142-56 systemd[1]: Starting Kubernetes Kubelet...
Jul 18 01:48:27.100439 ip-10-0-142-56 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:48:27.100439 ip-10-0-142-56 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 18 01:48:27.100439 ip-10-0-142-56 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:48:27.100439 ip-10-0-142-56 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 18 01:48:27.100439 ip-10-0-142-56 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:48:27.102252 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.102130    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 18 01:48:27.106289 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106274    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:27.106289 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106289    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106294    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106297    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106300    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106303    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106306    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106308    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106311    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106314    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106316    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106318    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106321    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106323    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106326    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106328    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106331    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106335    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106339    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106342    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:27.106349 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106345    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106349    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106351    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106354    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106358    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106361    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106364    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106366    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106369    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106372    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106374    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106377    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106380    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106382    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106386    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106390    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106393    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106396    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106400    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106402    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:27.106900 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106405    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106408    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106410    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106413    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106416    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106418    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106420    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106423    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106425    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106428    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106430    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106432    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106435    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106437    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106440    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106443    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106445    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106448    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106451    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106453    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:27.107403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106455    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106458    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106460    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106463    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106465    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106469    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106471    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106474    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106477    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106479    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106481    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106485    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106487    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106490    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106492    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106494    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106497    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106499    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106502    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106504    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:27.107909 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106507    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106509    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106512    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106514    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106516    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106519    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106523    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106525    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106528    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106530    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106533    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106535    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106537    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106540    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106543    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106547    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106550    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106552    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:27.108403 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.106555    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:27.108870 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108858    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:27.108870 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108870    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108874    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108877    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108879    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108882    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108885    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108888    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108890    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108894    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108898    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108900    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108903    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108906    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108908    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108911    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108921    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108925    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108927    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108930    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108932    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:27.108933 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108935    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108937    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108940    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108943    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108945    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108948    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108951    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108953    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108956    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108958    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108961    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108963    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108966    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108968    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108970    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108973    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108975    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108978    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108981    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108983    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:27.109437 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108986    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108988    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108990    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108993    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108995    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.108997    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109000    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109002    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109005    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109007    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109010    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109012    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109014    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109017    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109020    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109022    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109025    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109027    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109030    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109032    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:27.109972 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109035    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109052    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109057    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109060    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109063    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109066    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109069    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109071    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109074    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109076    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109080    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109082    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109085    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109088    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109091    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109094    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109096    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109099    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109101    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109104    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:27.110483 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109107    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109110    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109114    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109117    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109119    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109122    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109125    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109128    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109130    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109133    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109135    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109138    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109140    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109143    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109145    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109148    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109150    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109153    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109498    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:27.110974 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109503    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109505    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109508    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109516    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109519    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109522    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109524    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109527    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109529    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109532    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109535    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109537    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109541    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109545    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109549    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109552    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109555    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109557    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109560    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:27.111449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109563    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109566    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109569    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109571    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109573    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109576    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109579    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109581    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109584    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109586    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109589    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109591    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109594    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109596    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109598    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109601    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109603    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109606    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109609    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109611    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:27.111911 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109615    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109619    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109621    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109624    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109626    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109629    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109632    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109634    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109636    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109639    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109641    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109644    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109647    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109649    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109652    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109654    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109657    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109659    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109662    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109664    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:27.112436 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109667    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109669    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109673    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109676    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109679    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109682    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109685    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109688    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109691    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109694    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109697    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109699    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109702    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109704    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109707    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109709    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109711    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109714    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109717    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:27.112922 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109719    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109722    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109724    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109726    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109729    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109732    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109734    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109736    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109739    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109741    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109744    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109746    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109749    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109751    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109754    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109756    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109759    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109761    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109763    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.109766    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109840    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 18 01:48:27.113410 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109847    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109855    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109860    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109865    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109868    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109872    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109876    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109879    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109882    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109886    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109889    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109892    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109895    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109898    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109900    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109903    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109906    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109911    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109913    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109916    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109919    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109922    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109926    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 18 01:48:27.113910 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109929    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109932    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109935    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109937    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109940    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109943    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109946    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109949    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109953    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109956    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109959    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109961    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109964    2580 flags.go:64] FLAG: --enable-server="true"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109967    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109971    2580 flags.go:64] FLAG: --event-burst="100"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109974    2580 flags.go:64] FLAG: --event-qps="50"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109977    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109981    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109984    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109988    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109990    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109993    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109996    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.109999    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110002    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 18 01:48:27.114472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110005    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110008    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110011    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110013    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110016    2580 flags.go:64] FLAG: --feature-gates=""
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110020    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110023    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110025    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110029    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110031    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110034    2580 flags.go:64] FLAG: --help="false"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110037    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110055    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110058    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110061    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110064    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110068    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110070    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110073    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110076    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110080    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110083    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110086    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110089    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 18 01:48:27.115155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110092    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110094    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110098    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110101    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110103    2580 flags.go:64] FLAG: --lock-file=""
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110106    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110109    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110112    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110117    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110120    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110123    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110126    2580 flags.go:64] FLAG: --logging-format="text"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110128    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110131    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110134    2580 flags.go:64] FLAG: --manifest-url=""
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110137    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110141    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110144    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110148    2580 flags.go:64] FLAG: --max-pods="110"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110151    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110154    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110160    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110163    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110167    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110169    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 18 01:48:27.115739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110172    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110179    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110182    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110185    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110188    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110191    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110196    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110199    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110202    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110205    2580 flags.go:64] FLAG: --port="10250"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110208    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110211    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-092d8d04866291f19"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110214    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110217    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110219    2580 flags.go:64] FLAG: --register-node="true"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110223    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110226    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110229    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110231    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110234    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110237    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110240    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110243    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110246    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110249    2580 flags.go:64] FLAG: --runonce="false"
Jul 18 01:48:27.116396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110251    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110254    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110257    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110260    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110264    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110267    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110270    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110273    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110276    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110279    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110281    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110284    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110287    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110291    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110293    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110299    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110302    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110304    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110309    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110311    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110314    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110316    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110319    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110322    2580 flags.go:64] FLAG: --v="2"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110326    2580 flags.go:64] FLAG: --version="false"
Jul 18 01:48:27.117037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110330    2580 flags.go:64] FLAG: --vmodule=""
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110333    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110336    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110469    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110473    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110476    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110480    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110483    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110486    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110489    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110491    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110495    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110498    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110501    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110504    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110506    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110510    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110513    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110515    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:27.117649 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110518    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110520    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110523    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110526    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110528    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110531    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110533    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110535    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110539    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110542    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110545    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110548    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110551    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110553    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110556    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110558    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110562    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110565    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110568    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:27.118124 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110571    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110574    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110576    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110579    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110581    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110586    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110589    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110591    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110594    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110596    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110602    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110605    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110607    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110610    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110612    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110615    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110618    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110621    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110623    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110626    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:27.118587 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110628    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110631    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110633    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110636    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110639    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110641    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110644    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110646    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110648    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110651    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110653    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110656    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110658    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110660    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110663    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110665    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110668    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110672    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110674    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110677    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:27.119167 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110679    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110681    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110685    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110687    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110690    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110692    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110695    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110697    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110700    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110703    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110705    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110708    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110711    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110713    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110716    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110718    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110720    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110723    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110725    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110728    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:27.119644 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110730    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110733    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110735    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.110737    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.110742    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.118773    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.118791    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.118807    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.118812    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118872    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118877    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118880    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118883    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118886    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118889    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:27.120131 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118891    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118894    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118896    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118900    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118904    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118907    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118910    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118912    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118915    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118917    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118920    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118922    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118925    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118928    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118930    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118934    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118937    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118940    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118942    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:27.120507 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118945    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118948    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118950    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118953    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118962    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118965    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118968    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118970    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118972    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118975    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118977    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118980    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118983    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118987    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118989    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118993    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118995    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.118998    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119000    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:27.120975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119003    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119006    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119008    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119011    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119013    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119015    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119018    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119020    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119023    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119025    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119027    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119030    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119032    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119035    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119037    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119056    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119059    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119062    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119071    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119073    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:27.121452 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119076    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119078    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119081    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119083    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119086    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119088    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119090    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119093    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119096    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119098    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119101    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119104    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119106    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119108    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119111    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119114    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119116    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119118    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119121    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119123    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:27.121927 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119126    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119128    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119130    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119133    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119135    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119138    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119140    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119142    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119145    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119148    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119151    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119159    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119161    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119164    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119166    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:27.122449 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.119171    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119323    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119328    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119332    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119336    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119339    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119342    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119344    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119347    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119349    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119351    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119354    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119356    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119359    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119361    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119363    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119366    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119368    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119370    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:27.122824 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119373    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119375    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119377    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119380    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119382    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119385    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119387    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119390    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119393    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119395    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119404    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119407    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119409    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119411    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119414    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119416    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119419    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119421    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119424    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119426    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:27.123324 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119428    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119431    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119433    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119435    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119438    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119440    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119443    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119445    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119448    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119450    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119452    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119455    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119457    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119459    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119462    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119464    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119466    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119469    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119471    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119473    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:27.123813 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119476    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119479    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119481    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119489    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119492    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119494    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119497    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119499    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119501    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119504    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119506    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119508    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119511    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119513    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119516    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119518    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119520    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119523    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119525    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119527    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:27.124301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119530    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119532    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119535    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119537    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119540    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119542    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119545    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119549    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119552    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119554    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119557    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119559    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119562    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119564    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119568    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119571    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119579    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119583    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119585    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:27.124768 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119588    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:27.125263 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:27.119591    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:27.125263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.119596    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:48:27.125263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.119713    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 18 01:48:27.125263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.122590    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 18 01:48:27.125263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.124249    2580 server.go:1019] "Starting client certificate rotation"
Jul 18 01:48:27.125263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.124354    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:48:27.125263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.124387    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:48:27.127606 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.127592    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:48:27.131750 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.131733    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:48:27.145371 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.145347    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 18 01:48:27.147481 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.147465    2580 log.go:25] "Validated CRI v1 image API"
Jul 18 01:48:27.148308 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.148296    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 18 01:48:27.152162 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.152141    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 84b53e0e-decc-476e-91b7-1fde832bd576:/dev/nvme0n1p3 c193f5c0-198c-4237-9937-b4f0adab6a94:/dev/nvme0n1p4]
Jul 18 01:48:27.152214 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.152162    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 18 01:48:27.153511 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.153494    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:48:27.159247 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.159145    2580 manager.go:216] Machine: {Timestamp:2026-07-18 01:48:27.157818005 +0000 UTC m=+0.367532217 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098662 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21676f6088f6ee6f14d369fb630a42 SystemUUID:ec21676f-6088-f6ee-6f14-d369fb630a42 BootID:b8979665-510e-4616-8a26-32de319eea96 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7c:78:8e:71:43 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7c:78:8e:71:43 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:f3:42:c7:9f:b3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 18 01:48:27.159247 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.159243    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 18 01:48:27.159339 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.159316    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 18 01:48:27.162288 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.162263    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 18 01:48:27.162469 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.162291    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-56.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 18 01:48:27.163639 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.163629    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 18 01:48:27.163674 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.163642    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 18 01:48:27.163674 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.163649    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:48:27.163674 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.163664    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:48:27.164929 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.164911    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 18 01:48:27.168741 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.168730    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:48:27.168892 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.168883    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 18 01:48:27.168935 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.168911    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 18 01:48:27.168935 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.168930    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 18 01:48:27.168996 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.168948    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 18 01:48:27.168996 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.168956    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 18 01:48:27.168996 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.168965    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 18 01:48:27.169339 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.169327    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:48:27.169374 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.169345    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:48:27.171525 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.171503    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 18 01:48:27.173222 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.173205    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 18 01:48:27.173312 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.173244    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 18 01:48:27.175174 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175148    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 18 01:48:27.175174 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175170    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 18 01:48:27.175174 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175178    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 18 01:48:27.175352 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175186    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 18 01:48:27.175352 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175196    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 18 01:48:27.175352 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175204    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 18 01:48:27.175352 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175215    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 18 01:48:27.175352 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175235    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 18 01:48:27.175352 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175244    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 18 01:48:27.175352 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175252    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 18 01:48:27.175352 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175264    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 18 01:48:27.175352 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.175272    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 18 01:48:27.176159 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.176147    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 18 01:48:27.176213 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.176161    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 18 01:48:27.178644 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.178617    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dgsng"
Jul 18 01:48:27.179531 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.179501    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 18 01:48:27.179531 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.179520    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-56.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 18 01:48:27.179531 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.179512    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-56.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 18 01:48:27.179979 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.179962    2580 server.go:1268] "Started kubelet"
Jul 18 01:48:27.180343 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.180290    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 18 01:48:27.180442 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.180363    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 18 01:48:27.180705 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.180663    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 18 01:48:27.181105 ip-10-0-142-56 systemd[1]: Started Kubernetes Kubelet.
Jul 18 01:48:27.182026 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.181987    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 18 01:48:27.183630 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.183604    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 18 01:48:27.185098 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.185082    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dgsng"
Jul 18 01:48:27.187586 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.187567    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 18 01:48:27.189396 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.188533    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-56.ec2.internal.18c33f256339f8bf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-56.ec2.internal,UID:ip-10-0-142-56.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-56.ec2.internal,},FirstTimestamp:2026-07-18 01:48:27.179931839 +0000 UTC m=+0.389646051,LastTimestamp:2026-07-18 01:48:27.179931839 +0000 UTC m=+0.389646051,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-56.ec2.internal,}"
Jul 18 01:48:27.191181 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.191169    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 18 01:48:27.191261 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.191242    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 18 01:48:27.191313 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.191304    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:27.191790 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.191775    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 18 01:48:27.191790 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.191792    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 18 01:48:27.191932 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.191775    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 18 01:48:27.191932 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.191860    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 18 01:48:27.191932 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.191868    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 18 01:48:27.192097 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.192058    2580 factory.go:55] Registering systemd factory
Jul 18 01:48:27.192097 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.192071    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 18 01:48:27.192259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.192244    2580 factory.go:153] Registering CRI-O factory
Jul 18 01:48:27.192305 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.192266    2580 factory.go:223] Registration of the crio container factory successfully
Jul 18 01:48:27.192349 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.192314    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 18 01:48:27.192349 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.192337    2580 factory.go:103] Registering Raw factory
Jul 18 01:48:27.192441 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.192353    2580 manager.go:1196] Started watching for new ooms in manager
Jul 18 01:48:27.192856 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.192845    2580 manager.go:349] Starting recovery of all containers
Jul 18 01:48:27.201022 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.200956    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:27.202101 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.202086    2580 manager.go:354] Recovery completed
Jul 18 01:48:27.204628 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.204474    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-56.ec2.internal\" not found" node="ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.206643 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.206632    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:27.208409 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.208393    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:27.208467 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.208424    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:27.208467 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.208438    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:27.208911 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.208896    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 18 01:48:27.208911 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.208907    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:27.209059 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.208909    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 18 01:48:27.209059 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.208956    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:48:27.209556 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.209540    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:27.209638 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.209561    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:27.209638 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.209572    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:27.210070 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.210055    2580 policy_none.go:49] "None policy: Start"
Jul 18 01:48:27.210116 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.210074    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 18 01:48:27.210116 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.210084    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 18 01:48:27.211131 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.211120    2580 policy_none.go:47] "Start"
Jul 18 01:48:27.245602 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.245587    2580 manager.go:337] "Starting Device Plugin manager"
Jul 18 01:48:27.245684 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.245620    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 18 01:48:27.245684 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.245629    2580 server.go:87] "Starting device plugin registration server"
Jul 18 01:48:27.245866 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.245854    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 18 01:48:27.245926 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.245878    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 18 01:48:27.245999 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.245980    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 18 01:48:27.246097 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.246083    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 18 01:48:27.246097 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.246096    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 18 01:48:27.246623 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.246600    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 18 01:48:27.246694 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.246658    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:27.346270 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.346243    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:27.347350 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.347186    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:27.347350 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.347218    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:27.347350 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.347229    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:27.347350 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.347256    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.353175 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.353141    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.353175 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.353163    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-56.ec2.internal\": node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:27.366412 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.366390    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:27.383063 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.383026    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 18 01:48:27.384319 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.384304    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 18 01:48:27.384373 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.384347    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 18 01:48:27.384373 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.384371    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 18 01:48:27.384440 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.384400    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 18 01:48:27.386497 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.386481    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:27.466820 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.466792    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:27.484966 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.484933    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal"]
Jul 18 01:48:27.485020 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.485007    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:27.485883 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.485869    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:27.485937 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.485898    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:27.485937 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.485909    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:27.487185 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.487171    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:27.487330 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.487316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.487391 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.487351    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:27.487827 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.487814    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:27.487827 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.487824    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:27.487936 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.487838    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:27.487936 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.487843    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:27.487936 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.487852    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:27.488098 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.487856    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:27.489076 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.489059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.489176 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.489091    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:27.489732 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.489716    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:27.489778 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.489750    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:27.489778 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.489765    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:27.492724 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.492707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ee5ddcfe47f85210fa7b807fdc94725-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal\" (UID: \"9ee5ddcfe47f85210fa7b807fdc94725\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.492786 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.492730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/264b181dc92a9549d9a55f9f442fc196-config\") pod \"kube-apiserver-proxy-ip-10-0-142-56.ec2.internal\" (UID: \"264b181dc92a9549d9a55f9f442fc196\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.492786 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.492746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ee5ddcfe47f85210fa7b807fdc94725-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal\" (UID: \"9ee5ddcfe47f85210fa7b807fdc94725\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.513571 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.513548    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-56.ec2.internal\" not found" node="ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.517954 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.517939    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-56.ec2.internal\" not found" node="ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.567578 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.567557    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:27.592844 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.592822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ee5ddcfe47f85210fa7b807fdc94725-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal\" (UID: \"9ee5ddcfe47f85210fa7b807fdc94725\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.592921 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.592892    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ee5ddcfe47f85210fa7b807fdc94725-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal\" (UID: \"9ee5ddcfe47f85210fa7b807fdc94725\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.592921 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.592908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ee5ddcfe47f85210fa7b807fdc94725-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal\" (UID: \"9ee5ddcfe47f85210fa7b807fdc94725\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.592986 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.592849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ee5ddcfe47f85210fa7b807fdc94725-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal\" (UID: \"9ee5ddcfe47f85210fa7b807fdc94725\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.592986 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.592955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/264b181dc92a9549d9a55f9f442fc196-config\") pod \"kube-apiserver-proxy-ip-10-0-142-56.ec2.internal\" (UID: \"264b181dc92a9549d9a55f9f442fc196\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.592986 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.592979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/264b181dc92a9549d9a55f9f442fc196-config\") pod \"kube-apiserver-proxy-ip-10-0-142-56.ec2.internal\" (UID: \"264b181dc92a9549d9a55f9f442fc196\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.668178 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.668105    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:27.768703 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.768660    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:27.817195 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.817166    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.821311 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:27.821085    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:27.868825 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.868795    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:27.969623 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:27.969540    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:28.070227 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:28.070188    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:28.124725 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.124697    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 18 01:48:28.125175 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.124861    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 01:48:28.125175 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.124902    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 01:48:28.171919 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:28.171886    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:28.190733 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.190697    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 01:43:28 +0000 UTC" deadline="2028-01-07 17:58:47.457636135 +0000 UTC"
Jul 18 01:48:28.190733 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.190725    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12928h10m19.26691511s"
Jul 18 01:48:28.272288 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:28.272254    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-56.ec2.internal\" not found"
Jul 18 01:48:28.295108 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.295075    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:28.344293 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:28.344247    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264b181dc92a9549d9a55f9f442fc196.slice/crio-671c5edd26b05d40d0cb07a30309df4eae23cfbabc0f97bd1295aef4687d2e5f WatchSource:0}: Error finding container 671c5edd26b05d40d0cb07a30309df4eae23cfbabc0f97bd1295aef4687d2e5f: Status 404 returned error can't find the container with id 671c5edd26b05d40d0cb07a30309df4eae23cfbabc0f97bd1295aef4687d2e5f
Jul 18 01:48:28.344525 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:28.344505    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee5ddcfe47f85210fa7b807fdc94725.slice/crio-cfa4e36c41c304110dd2646dfadc12c62a99ed0d4e32d5761012b1d515c0fb18 WatchSource:0}: Error finding container cfa4e36c41c304110dd2646dfadc12c62a99ed0d4e32d5761012b1d515c0fb18: Status 404 returned error can't find the container with id cfa4e36c41c304110dd2646dfadc12c62a99ed0d4e32d5761012b1d515c0fb18
Jul 18 01:48:28.348735 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.348717    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:48:28.387797 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.387749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal" event={"ID":"264b181dc92a9549d9a55f9f442fc196","Type":"ContainerStarted","Data":"671c5edd26b05d40d0cb07a30309df4eae23cfbabc0f97bd1295aef4687d2e5f"}
Jul 18 01:48:28.388908 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.388886    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal" event={"ID":"9ee5ddcfe47f85210fa7b807fdc94725","Type":"ContainerStarted","Data":"cfa4e36c41c304110dd2646dfadc12c62a99ed0d4e32d5761012b1d515c0fb18"}
Jul 18 01:48:28.392302 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.392286    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:28.404886 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.404870    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:48:28.406999 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.406983    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal"
Jul 18 01:48:28.416095 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.416080    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:48:28.593407 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:28.593317    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:29.172912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.172875    2580 apiserver.go:52] "Watching apiserver"
Jul 18 01:48:29.184032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.184002    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 18 01:48:29.184522 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.184489    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-b9gn8","kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6","openshift-cluster-node-tuning-operator/tuned-pmjvt","openshift-multus/multus-additional-cni-plugins-88dq9","openshift-multus/multus-jm445","openshift-network-diagnostics/network-check-target-lxbgv","openshift-dns/node-resolver-jk4xm","openshift-image-registry/node-ca-lpmsm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal","openshift-multus/network-metrics-daemon-npj5j","openshift-network-operator/iptables-alerter-drjsv","openshift-ovn-kubernetes/ovnkube-node-qmfpr","kube-system/global-pull-secret-syncer-qj8km"]
Jul 18 01:48:29.186107 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.186085    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:29.186205 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.186160    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:29.188196 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.188172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.189502 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.189355    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.190840 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.190806    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.191080 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.191049    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 18 01:48:29.191399 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.191383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 18 01:48:29.191649 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.191629    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p7cgk\""
Jul 18 01:48:29.191649 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.191648    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 18 01:48:29.191813 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.191798    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 18 01:48:29.191903 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.191885    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 18 01:48:29.193096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.192587    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 18 01:48:29.193096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.192649    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 18 01:48:29.193096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.192741    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dmbfz\""
Jul 18 01:48:29.193096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.192821    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 18 01:48:29.193096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.192590    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 18 01:48:29.193096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.192912    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 18 01:48:29.193096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.193060    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.193931 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.193909    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:48:29.194026 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.194007    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pl4mr\""
Jul 18 01:48:29.194210 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.194179    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 18 01:48:29.194505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.194488    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.197460 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.196392    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 18 01:48:29.197460 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.196681    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 18 01:48:29.197460 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.197005    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 18 01:48:29.197460 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.197071    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 18 01:48:29.197460 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.197366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.197460 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.197390    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 18 01:48:29.199400 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.198892    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:29.199547 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.199523    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 18 01:48:29.200175 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.196954    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fnm9l\""
Jul 18 01:48:29.200967 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.200922    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h7j7q\""
Jul 18 01:48:29.202052 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-hostroot\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.202180 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-systemd-units\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.202180 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202101    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 18 01:48:29.202180 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/934531a2-afe2-4bd6-ba63-3d716fbc8f15-ovnkube-config\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.202180 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c7sm\" (UniqueName: \"kubernetes.io/projected/8dc3c51b-dafb-4e39-952e-ab0222934405-kube-api-access-8c7sm\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.202439 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-os-release\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.202439 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-etc-openvswitch\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.202439 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-node-log\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.202439 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-kubernetes\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.202439 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202397    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-systemd\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.202439 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202431    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-cnibin\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8222da46-c8c4-4d81-b240-a4455631215c-multus-daemon-config\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202494    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-cni-bin\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-socket-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-sysctl-d\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-var-lib-kubelet\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d8131089-a28e-46d5-b800-de5c8e83457b-cni-binary-copy\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-run-netns\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/934531a2-afe2-4bd6-ba63-3d716fbc8f15-ovn-node-metrics-cert\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-host\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.202761 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-cnibin\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-run-netns\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d8131089-a28e-46d5-b800-de5c8e83457b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-multus-socket-dir-parent\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.202995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-run-k8s-cni-cncf-io\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-run-ovn\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-sysctl-conf\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-sys\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-var-lib-cni-bin\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-device-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-tuned\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203262    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-os-release\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-run-systemd\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.203385 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.203348    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-var-lib-cni-multus\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203472    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203538    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/934531a2-afe2-4bd6-ba63-3d716fbc8f15-env-overrides\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203625    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-sysconfig\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-run\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-var-lib-kubelet\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-sys-fs\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.203977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25hk8\" (UniqueName: \"kubernetes.io/projected/082d9fe7-b05b-448e-973d-a309eaac5b57-kube-api-access-25hk8\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dc3c51b-dafb-4e39-952e-ab0222934405-tmp\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5k2k\" (UniqueName: \"kubernetes.io/projected/d8131089-a28e-46d5-b800-de5c8e83457b-kube-api-access-b5k2k\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.204158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-etc-selinux\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-modprobe-d\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204204    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-lib-modules\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-system-cni-dir\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b4mf\" (UniqueName: \"kubernetes.io/projected/8222da46-c8c4-4d81-b240-a4455631215c-kube-api-access-7b4mf\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-slash\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-var-lib-openvswitch\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-run-openvswitch\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204458    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-multus-conf-dir\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-etc-kubernetes\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-cni-netd\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d8131089-a28e-46d5-b800-de5c8e83457b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-multus-cni-dir\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8222da46-c8c4-4d81-b240-a4455631215c-cni-binary-copy\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-run-multus-certs\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.204918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-kubelet\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.205798 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.205798 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.204957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/934531a2-afe2-4bd6-ba63-3d716fbc8f15-ovnkube-script-lib\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.205798 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.205639    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 18 01:48:29.205798 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.205752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-system-cni-dir\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.205798 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.205792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-log-socket\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.206212 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.205878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2t47\" (UniqueName: \"kubernetes.io/projected/934531a2-afe2-4bd6-ba63-3d716fbc8f15-kube-api-access-b2t47\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.206212 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.206043    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qffkh\""
Jul 18 01:48:29.206212 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.206154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-registration-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.206390 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.206263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zxhpn\""
Jul 18 01:48:29.208885 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.208826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.209219 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.209200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:29.209317 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.209259    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:29.210912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.209821    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 18 01:48:29.210912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.210071    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 18 01:48:29.210912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.210356    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r4d6w\""
Jul 18 01:48:29.210912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.210630    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:48:29.213143 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.213090    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:48:29.213287 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.213149    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 18 01:48:29.213610 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.213564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-llt6v\""
Jul 18 01:48:29.213852 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.213835    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 18 01:48:29.235170 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.235135    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dw2tl"
Jul 18 01:48:29.245795 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.245768    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dw2tl"
Jul 18 01:48:29.270580 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.270519    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:29.293371 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.293349    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 18 01:48:29.306969 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.306934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c7sm\" (UniqueName: \"kubernetes.io/projected/8dc3c51b-dafb-4e39-952e-ab0222934405-kube-api-access-8c7sm\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.307104 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.306979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-os-release\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.307104 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-etc-openvswitch\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.307104 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bc9c16a-612f-4d70-a673-79af0c13a275-tmp-dir\") pod \"node-resolver-jk4xm\" (UID: \"8bc9c16a-612f-4d70-a673-79af0c13a275\") " pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.307104 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/108086ce-a8d1-46d1-904d-86c21e823ad7-host\") pod \"node-ca-lpmsm\" (UID: \"108086ce-a8d1-46d1-904d-86c21e823ad7\") " pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.307351 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-node-log\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.307351 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-node-log\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.307351 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-etc-openvswitch\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.307351 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-kubernetes\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.307351 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-os-release\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.307351 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307296    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-systemd\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.307351 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-cnibin\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-systemd\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8222da46-c8c4-4d81-b240-a4455631215c-multus-daemon-config\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-cni-bin\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-cnibin\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-kubernetes\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-socket-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lsxq\" (UniqueName: \"kubernetes.io/projected/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-kube-api-access-4lsxq\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b0bef1c-ce07-4823-a070-285f60a843e5-host-slash\") pod \"iptables-alerter-drjsv\" (UID: \"9b0bef1c-ce07-4823-a070-285f60a843e5\") " pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-socket-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-sysctl-d\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-var-lib-kubelet\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-cni-bin\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d8131089-a28e-46d5-b800-de5c8e83457b-cni-binary-copy\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-run-netns\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/934531a2-afe2-4bd6-ba63-3d716fbc8f15-ovn-node-metrics-cert\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:29.307726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307709    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-host\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-cnibin\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-sysctl-d\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-run-netns\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqv7d\" (UniqueName: \"kubernetes.io/projected/8bc9c16a-612f-4d70-a673-79af0c13a275-kube-api-access-dqv7d\") pod \"node-resolver-jk4xm\" (UID: \"8bc9c16a-612f-4d70-a673-79af0c13a275\") " pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-var-lib-kubelet\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d8131089-a28e-46d5-b800-de5c8e83457b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307893    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-multus-socket-dir-parent\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-run-k8s-cni-cncf-io\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8222da46-c8c4-4d81-b240-a4455631215c-multus-daemon-config\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-run-ovn\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.307992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-sysctl-conf\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-run-netns\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-sys\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-var-lib-cni-bin\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-sys\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-device-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-var-lib-cni-bin\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.308679 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308146    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-device-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/108086ce-a8d1-46d1-904d-86c21e823ad7-serviceca\") pod \"node-ca-lpmsm\" (UID: \"108086ce-a8d1-46d1-904d-86c21e823ad7\") " pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-tuned\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-run-k8s-cni-cncf-io\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-cnibin\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-os-release\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-run-netns\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-run-systemd\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d8131089-a28e-46d5-b800-de5c8e83457b-cni-binary-copy\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8bc9c16a-612f-4d70-a673-79af0c13a275-hosts-file\") pod \"node-resolver-jk4xm\" (UID: \"8bc9c16a-612f-4d70-a673-79af0c13a275\") " pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zbzz\" (UniqueName: \"kubernetes.io/projected/9b0bef1c-ce07-4823-a070-285f60a843e5-kube-api-access-9zbzz\") pod \"iptables-alerter-drjsv\" (UID: \"9b0bef1c-ce07-4823-a070-285f60a843e5\") " pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-var-lib-cni-multus\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/934531a2-afe2-4bd6-ba63-3d716fbc8f15-env-overrides\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308424    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-sysctl-conf\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-sysconfig\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-run\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-sysconfig\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.309346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-var-lib-kubelet\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-run-systemd\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-var-lib-cni-multus\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-run-ovn\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308564    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308632    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-run\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-sys-fs\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-os-release\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25hk8\" (UniqueName: \"kubernetes.io/projected/082d9fe7-b05b-448e-973d-a309eaac5b57-kube-api-access-25hk8\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-host\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/42e00b16-b4d4-4e92-8a85-b4477120c978-dbus\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dc3c51b-dafb-4e39-952e-ab0222934405-tmp\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5k2k\" (UniqueName: \"kubernetes.io/projected/d8131089-a28e-46d5-b800-de5c8e83457b-kube-api-access-b5k2k\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-etc-selinux\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/42e00b16-b4d4-4e92-8a85-b4477120c978-kubelet-config\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.310015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9b0bef1c-ce07-4823-a070-285f60a843e5-iptables-alerter-script\") pod \"iptables-alerter-drjsv\" (UID: \"9b0bef1c-ce07-4823-a070-285f60a843e5\") " pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-modprobe-d\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-lib-modules\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-multus-socket-dir-parent\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-system-cni-dir\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b4mf\" (UniqueName: \"kubernetes.io/projected/8222da46-c8c4-4d81-b240-a4455631215c-kube-api-access-7b4mf\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-sys-fs\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-slash\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-var-lib-openvswitch\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-run-openvswitch\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3bc46aff-98b6-4d1d-81eb-eaa4022de5b2-agent-certs\") pod \"konnectivity-agent-b9gn8\" (UID: \"3bc46aff-98b6-4d1d-81eb-eaa4022de5b2\") " pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309359    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d8131089-a28e-46d5-b800-de5c8e83457b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309417    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-var-lib-kubelet\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-modprobe-d\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.308948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/934531a2-afe2-4bd6-ba63-3d716fbc8f15-env-overrides\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.310913 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-system-cni-dir\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-run-openvswitch\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-slash\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-var-lib-openvswitch\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-multus-conf-dir\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-etc-kubernetes\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309705    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-multus-conf-dir\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-cni-netd\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj268\" (UniqueName: \"kubernetes.io/projected/108086ce-a8d1-46d1-904d-86c21e823ad7-kube-api-access-vj268\") pod \"node-ca-lpmsm\" (UID: \"108086ce-a8d1-46d1-904d-86c21e823ad7\") " pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309756    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-etc-selinux\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8dc3c51b-dafb-4e39-952e-ab0222934405-lib-modules\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d8131089-a28e-46d5-b800-de5c8e83457b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309916    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-etc-kubernetes\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-multus-cni-dir\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.309972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-cni-netd\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8222da46-c8c4-4d81-b240-a4455631215c-cni-binary-copy\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d8131089-a28e-46d5-b800-de5c8e83457b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.311635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-multus-cni-dir\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-run-multus-certs\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-kubelet\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/934531a2-afe2-4bd6-ba63-3d716fbc8f15-ovnkube-script-lib\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-system-cni-dir\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-log-socket\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-system-cni-dir\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d8131089-a28e-46d5-b800-de5c8e83457b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-host-kubelet\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-host-run-multus-certs\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2t47\" (UniqueName: \"kubernetes.io/projected/934531a2-afe2-4bd6-ba63-3d716fbc8f15-kube-api-access-b2t47\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-log-socket\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310501    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-registration-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8222da46-c8c4-4d81-b240-a4455631215c-cni-binary-copy\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.312589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/082d9fe7-b05b-448e-973d-a309eaac5b57-registration-dir\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.313496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-hostroot\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.313496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-systemd-units\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.313496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8222da46-c8c4-4d81-b240-a4455631215c-hostroot\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.313496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/934531a2-afe2-4bd6-ba63-3d716fbc8f15-ovnkube-config\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.313496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/934531a2-afe2-4bd6-ba63-3d716fbc8f15-systemd-units\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.313496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3bc46aff-98b6-4d1d-81eb-eaa4022de5b2-konnectivity-ca\") pod \"konnectivity-agent-b9gn8\" (UID: \"3bc46aff-98b6-4d1d-81eb-eaa4022de5b2\") " pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:29.313496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.310942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/934531a2-afe2-4bd6-ba63-3d716fbc8f15-ovnkube-script-lib\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.313496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.311151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/934531a2-afe2-4bd6-ba63-3d716fbc8f15-ovnkube-config\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.313496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.312597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8dc3c51b-dafb-4e39-952e-ab0222934405-etc-tuned\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.313932 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.313636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/934531a2-afe2-4bd6-ba63-3d716fbc8f15-ovn-node-metrics-cert\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.314152 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.314112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dc3c51b-dafb-4e39-952e-ab0222934405-tmp\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.317092 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.316335    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:29.317092 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.316390    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:29.317092 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.316409    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tvshg for pod openshift-network-diagnostics/network-check-target-lxbgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:29.317092 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.316506    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg podName:56718ed9-ac2a-4497-8484-20b905ba2022 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:29.816468801 +0000 UTC m=+3.026183022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tvshg" (UniqueName: "kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg") pod "network-check-target-lxbgv" (UID: "56718ed9-ac2a-4497-8484-20b905ba2022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:29.317387 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.317252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c7sm\" (UniqueName: \"kubernetes.io/projected/8dc3c51b-dafb-4e39-952e-ab0222934405-kube-api-access-8c7sm\") pod \"tuned-pmjvt\" (UID: \"8dc3c51b-dafb-4e39-952e-ab0222934405\") " pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.319664 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.319639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5k2k\" (UniqueName: \"kubernetes.io/projected/d8131089-a28e-46d5-b800-de5c8e83457b-kube-api-access-b5k2k\") pod \"multus-additional-cni-plugins-88dq9\" (UID: \"d8131089-a28e-46d5-b800-de5c8e83457b\") " pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.320589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.320541    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b4mf\" (UniqueName: \"kubernetes.io/projected/8222da46-c8c4-4d81-b240-a4455631215c-kube-api-access-7b4mf\") pod \"multus-jm445\" (UID: \"8222da46-c8c4-4d81-b240-a4455631215c\") " pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.320676 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.320638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2t47\" (UniqueName: \"kubernetes.io/projected/934531a2-afe2-4bd6-ba63-3d716fbc8f15-kube-api-access-b2t47\") pod \"ovnkube-node-qmfpr\" (UID: \"934531a2-afe2-4bd6-ba63-3d716fbc8f15\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.323327 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.323297    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25hk8\" (UniqueName: \"kubernetes.io/projected/082d9fe7-b05b-448e-973d-a309eaac5b57-kube-api-access-25hk8\") pod \"aws-ebs-csi-driver-node-2hgn6\" (UID: \"082d9fe7-b05b-448e-973d-a309eaac5b57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.411361 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/42e00b16-b4d4-4e92-8a85-b4477120c978-dbus\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.411361 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/42e00b16-b4d4-4e92-8a85-b4477120c978-kubelet-config\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.411605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9b0bef1c-ce07-4823-a070-285f60a843e5-iptables-alerter-script\") pod \"iptables-alerter-drjsv\" (UID: \"9b0bef1c-ce07-4823-a070-285f60a843e5\") " pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.411605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3bc46aff-98b6-4d1d-81eb-eaa4022de5b2-agent-certs\") pod \"konnectivity-agent-b9gn8\" (UID: \"3bc46aff-98b6-4d1d-81eb-eaa4022de5b2\") " pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:29.411605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj268\" (UniqueName: \"kubernetes.io/projected/108086ce-a8d1-46d1-904d-86c21e823ad7-kube-api-access-vj268\") pod \"node-ca-lpmsm\" (UID: \"108086ce-a8d1-46d1-904d-86c21e823ad7\") " pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.411605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/42e00b16-b4d4-4e92-8a85-b4477120c978-dbus\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.411605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.411605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/42e00b16-b4d4-4e92-8a85-b4477120c978-kubelet-config\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.411605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3bc46aff-98b6-4d1d-81eb-eaa4022de5b2-konnectivity-ca\") pod \"konnectivity-agent-b9gn8\" (UID: \"3bc46aff-98b6-4d1d-81eb-eaa4022de5b2\") " pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:29.411605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bc9c16a-612f-4d70-a673-79af0c13a275-tmp-dir\") pod \"node-resolver-jk4xm\" (UID: \"8bc9c16a-612f-4d70-a673-79af0c13a275\") " pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.411605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/108086ce-a8d1-46d1-904d-86c21e823ad7-host\") pod \"node-ca-lpmsm\" (UID: \"108086ce-a8d1-46d1-904d-86c21e823ad7\") " pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lsxq\" (UniqueName: \"kubernetes.io/projected/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-kube-api-access-4lsxq\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.411657    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b0bef1c-ce07-4823-a070-285f60a843e5-host-slash\") pod \"iptables-alerter-drjsv\" (UID: \"9b0bef1c-ce07-4823-a070-285f60a843e5\") " pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.411736    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret podName:42e00b16-b4d4-4e92-8a85-b4477120c978 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:29.911714947 +0000 UTC m=+3.121429154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret") pod "global-pull-secret-syncer-qj8km" (UID: "42e00b16-b4d4-4e92-8a85-b4477120c978") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqv7d\" (UniqueName: \"kubernetes.io/projected/8bc9c16a-612f-4d70-a673-79af0c13a275-kube-api-access-dqv7d\") pod \"node-resolver-jk4xm\" (UID: \"8bc9c16a-612f-4d70-a673-79af0c13a275\") " pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.411778    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/108086ce-a8d1-46d1-904d-86c21e823ad7-serviceca\") pod \"node-ca-lpmsm\" (UID: \"108086ce-a8d1-46d1-904d-86c21e823ad7\") " pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.411827    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs podName:46d91c18-6dcf-4cb4-93a8-38d06290aa0a nodeName:}" failed. No retries permitted until 2026-07-18 01:48:29.911813307 +0000 UTC m=+3.121527522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs") pod "network-metrics-daemon-npj5j" (UID: "46d91c18-6dcf-4cb4-93a8-38d06290aa0a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8bc9c16a-612f-4d70-a673-79af0c13a275-hosts-file\") pod \"node-resolver-jk4xm\" (UID: \"8bc9c16a-612f-4d70-a673-79af0c13a275\") " pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bc9c16a-612f-4d70-a673-79af0c13a275-tmp-dir\") pod \"node-resolver-jk4xm\" (UID: \"8bc9c16a-612f-4d70-a673-79af0c13a275\") " pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zbzz\" (UniqueName: \"kubernetes.io/projected/9b0bef1c-ce07-4823-a070-285f60a843e5-kube-api-access-9zbzz\") pod \"iptables-alerter-drjsv\" (UID: \"9b0bef1c-ce07-4823-a070-285f60a843e5\") " pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.411961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/108086ce-a8d1-46d1-904d-86c21e823ad7-host\") pod \"node-ca-lpmsm\" (UID: \"108086ce-a8d1-46d1-904d-86c21e823ad7\") " pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.412017    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b0bef1c-ce07-4823-a070-285f60a843e5-host-slash\") pod \"iptables-alerter-drjsv\" (UID: \"9b0bef1c-ce07-4823-a070-285f60a843e5\") " pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.412064    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9b0bef1c-ce07-4823-a070-285f60a843e5-iptables-alerter-script\") pod \"iptables-alerter-drjsv\" (UID: \"9b0bef1c-ce07-4823-a070-285f60a843e5\") " pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.412154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.412070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8bc9c16a-612f-4d70-a673-79af0c13a275-hosts-file\") pod \"node-resolver-jk4xm\" (UID: \"8bc9c16a-612f-4d70-a673-79af0c13a275\") " pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.412995 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.412227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3bc46aff-98b6-4d1d-81eb-eaa4022de5b2-konnectivity-ca\") pod \"konnectivity-agent-b9gn8\" (UID: \"3bc46aff-98b6-4d1d-81eb-eaa4022de5b2\") " pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:29.412995 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.412799    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/108086ce-a8d1-46d1-904d-86c21e823ad7-serviceca\") pod \"node-ca-lpmsm\" (UID: \"108086ce-a8d1-46d1-904d-86c21e823ad7\") " pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.414405 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.414384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3bc46aff-98b6-4d1d-81eb-eaa4022de5b2-agent-certs\") pod \"konnectivity-agent-b9gn8\" (UID: \"3bc46aff-98b6-4d1d-81eb-eaa4022de5b2\") " pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:29.422615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.422587    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zbzz\" (UniqueName: \"kubernetes.io/projected/9b0bef1c-ce07-4823-a070-285f60a843e5-kube-api-access-9zbzz\") pod \"iptables-alerter-drjsv\" (UID: \"9b0bef1c-ce07-4823-a070-285f60a843e5\") " pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.422893 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.422866    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lsxq\" (UniqueName: \"kubernetes.io/projected/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-kube-api-access-4lsxq\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:29.423099 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.423038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqv7d\" (UniqueName: \"kubernetes.io/projected/8bc9c16a-612f-4d70-a673-79af0c13a275-kube-api-access-dqv7d\") pod \"node-resolver-jk4xm\" (UID: \"8bc9c16a-612f-4d70-a673-79af0c13a275\") " pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.423947 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.423926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj268\" (UniqueName: \"kubernetes.io/projected/108086ce-a8d1-46d1-904d-86c21e823ad7-kube-api-access-vj268\") pod \"node-ca-lpmsm\" (UID: \"108086ce-a8d1-46d1-904d-86c21e823ad7\") " pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.509418 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.509391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:29.517968 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.517940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6"
Jul 18 01:48:29.533140 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.533114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pmjvt"
Jul 18 01:48:29.539918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.539719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-88dq9"
Jul 18 01:48:29.549075 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.549040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jm445"
Jul 18 01:48:29.556859 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.556837    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lpmsm"
Jul 18 01:48:29.563846 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.563821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:29.565005 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.564985    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:29.573201 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.573178    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jk4xm"
Jul 18 01:48:29.580972 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.580949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-drjsv"
Jul 18 01:48:29.916005 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.915918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:29.916005 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.915971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:29.916243 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:29.916011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:29.916243 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.916096    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:29.916243 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.916157    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret podName:42e00b16-b4d4-4e92-8a85-b4477120c978 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:30.916136942 +0000 UTC m=+4.125851142 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret") pod "global-pull-secret-syncer-qj8km" (UID: "42e00b16-b4d4-4e92-8a85-b4477120c978") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:29.916243 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.916159    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:29.916243 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.916183    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:29.916243 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.916195    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tvshg for pod openshift-network-diagnostics/network-check-target-lxbgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:29.916243 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.916207    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:29.916243 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.916248    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg podName:56718ed9-ac2a-4497-8484-20b905ba2022 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:30.916232079 +0000 UTC m=+4.125946298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvshg" (UniqueName: "kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg") pod "network-check-target-lxbgv" (UID: "56718ed9-ac2a-4497-8484-20b905ba2022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:29.916533 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:29.916270    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs podName:46d91c18-6dcf-4cb4-93a8-38d06290aa0a nodeName:}" failed. No retries permitted until 2026-07-18 01:48:30.916259001 +0000 UTC m=+4.125973209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs") pod "network-metrics-daemon-npj5j" (UID: "46d91c18-6dcf-4cb4-93a8-38d06290aa0a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:30.248187 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:30.248105    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:43:30 +0000 UTC" deadline="2028-01-08 10:42:02.889821728 +0000 UTC"
Jul 18 01:48:30.248187 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:30.248143    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12944h53m32.641683636s"
Jul 18 01:48:30.428726 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:30.428676    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc46aff_98b6_4d1d_81eb_eaa4022de5b2.slice/crio-af9a9cd805133bf0932aedb4fff5346921ffa7f59efdce0107053638f382b27d WatchSource:0}: Error finding container af9a9cd805133bf0932aedb4fff5346921ffa7f59efdce0107053638f382b27d: Status 404 returned error can't find the container with id af9a9cd805133bf0932aedb4fff5346921ffa7f59efdce0107053638f382b27d
Jul 18 01:48:30.430328 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:30.430299    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0bef1c_ce07_4823_a070_285f60a843e5.slice/crio-61bbd072a7a940f5d25c3dd4c9c0d1741664a15caba697974524695b0a1d7fca WatchSource:0}: Error finding container 61bbd072a7a940f5d25c3dd4c9c0d1741664a15caba697974524695b0a1d7fca: Status 404 returned error can't find the container with id 61bbd072a7a940f5d25c3dd4c9c0d1741664a15caba697974524695b0a1d7fca
Jul 18 01:48:30.431367 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:30.431316    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108086ce_a8d1_46d1_904d_86c21e823ad7.slice/crio-2096ca5af0380624145410e618d86aafa703e890a53cb793e424e0c5eab9b5e5 WatchSource:0}: Error finding container 2096ca5af0380624145410e618d86aafa703e890a53cb793e424e0c5eab9b5e5: Status 404 returned error can't find the container with id 2096ca5af0380624145410e618d86aafa703e890a53cb793e424e0c5eab9b5e5
Jul 18 01:48:30.432284 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:30.432265    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod082d9fe7_b05b_448e_973d_a309eaac5b57.slice/crio-8501d721eb41bbb986c8b6ea515c38d00676a80f9d748ddbe7c86783b95be5aa WatchSource:0}: Error finding container 8501d721eb41bbb986c8b6ea515c38d00676a80f9d748ddbe7c86783b95be5aa: Status 404 returned error can't find the container with id 8501d721eb41bbb986c8b6ea515c38d00676a80f9d748ddbe7c86783b95be5aa
Jul 18 01:48:30.433285 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:30.433252    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc9c16a_612f_4d70_a673_79af0c13a275.slice/crio-683ff0b627011f7ad92b96ae8c7f1743cc2708ffd06147546d964a960ba0231b WatchSource:0}: Error finding container 683ff0b627011f7ad92b96ae8c7f1743cc2708ffd06147546d964a960ba0231b: Status 404 returned error can't find the container with id 683ff0b627011f7ad92b96ae8c7f1743cc2708ffd06147546d964a960ba0231b
Jul 18 01:48:30.434830 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:30.434809    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934531a2_afe2_4bd6_ba63_3d716fbc8f15.slice/crio-5e9168fee1a205153c52c83fc0f39acc539893dff14a154ef89563aba962cd5a WatchSource:0}: Error finding container 5e9168fee1a205153c52c83fc0f39acc539893dff14a154ef89563aba962cd5a: Status 404 returned error can't find the container with id 5e9168fee1a205153c52c83fc0f39acc539893dff14a154ef89563aba962cd5a
Jul 18 01:48:30.448706 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:30.448680    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8222da46_c8c4_4d81_b240_a4455631215c.slice/crio-d090a4b0c0c1dc3cd63b118e68f19451d8935f3d0a5a612024668dcd077a5eea WatchSource:0}: Error finding container d090a4b0c0c1dc3cd63b118e68f19451d8935f3d0a5a612024668dcd077a5eea: Status 404 returned error can't find the container with id d090a4b0c0c1dc3cd63b118e68f19451d8935f3d0a5a612024668dcd077a5eea
Jul 18 01:48:30.452975 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:48:30.452950    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc3c51b_dafb_4e39_952e_ab0222934405.slice/crio-d573c99e714bd1a59f9c6f4e9dfcb4564adf0c6febdcb569fd3a420ffcb8d53a WatchSource:0}: Error finding container d573c99e714bd1a59f9c6f4e9dfcb4564adf0c6febdcb569fd3a420ffcb8d53a: Status 404 returned error can't find the container with id d573c99e714bd1a59f9c6f4e9dfcb4564adf0c6febdcb569fd3a420ffcb8d53a
Jul 18 01:48:30.924549 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:30.924467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:30.924549 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:30.924532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:30.924765 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:30.924590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:30.924765 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:30.924707    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:30.924884 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:30.924768    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret podName:42e00b16-b4d4-4e92-8a85-b4477120c978 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:32.924748353 +0000 UTC m=+6.134462562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret") pod "global-pull-secret-syncer-qj8km" (UID: "42e00b16-b4d4-4e92-8a85-b4477120c978") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:30.925279 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:30.925259    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:30.925403 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:30.925284    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:30.925403 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:30.925299    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tvshg for pod openshift-network-diagnostics/network-check-target-lxbgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:30.925403 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:30.925350    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg podName:56718ed9-ac2a-4497-8484-20b905ba2022 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:32.925332768 +0000 UTC m=+6.135046971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvshg" (UniqueName: "kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg") pod "network-check-target-lxbgv" (UID: "56718ed9-ac2a-4497-8484-20b905ba2022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:30.925586 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:30.925415    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:30.925586 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:30.925450    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs podName:46d91c18-6dcf-4cb4-93a8-38d06290aa0a nodeName:}" failed. No retries permitted until 2026-07-18 01:48:32.925437756 +0000 UTC m=+6.135151971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs") pod "network-metrics-daemon-npj5j" (UID: "46d91c18-6dcf-4cb4-93a8-38d06290aa0a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:31.249269 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.249129    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:43:30 +0000 UTC" deadline="2028-03-25 19:49:44.018556272 +0000 UTC"
Jul 18 01:48:31.249269 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.249192    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14802h1m12.769369058s"
Jul 18 01:48:31.385619 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.385584    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:31.385820 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.385670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:31.385820 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.385689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:31.385928 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:31.385809    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:31.386236 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:31.386106    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:31.386236 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:31.386205    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:31.416656 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.415658    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ee5ddcfe47f85210fa7b807fdc94725" containerID="b63c0fcf3d51f5d28cd4eb18b28c9766914c193aabf447ff66ef56143ac7f97d" exitCode=0
Jul 18 01:48:31.416656 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.415751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal" event={"ID":"9ee5ddcfe47f85210fa7b807fdc94725","Type":"ContainerDied","Data":"b63c0fcf3d51f5d28cd4eb18b28c9766914c193aabf447ff66ef56143ac7f97d"}
Jul 18 01:48:31.438701 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.438668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal" event={"ID":"264b181dc92a9549d9a55f9f442fc196","Type":"ContainerStarted","Data":"d672e3e2710c5444ec3ffd136f8187127a16d607d964ac489ff1caec10de536c"}
Jul 18 01:48:31.455986 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.455925    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-56.ec2.internal" podStartSLOduration=2.455903547 podStartE2EDuration="2.455903547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:31.455361952 +0000 UTC m=+4.665076176" watchObservedRunningTime="2026-07-18 01:48:31.455903547 +0000 UTC m=+4.665617774"
Jul 18 01:48:31.460292 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.460261    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pmjvt" event={"ID":"8dc3c51b-dafb-4e39-952e-ab0222934405","Type":"ContainerStarted","Data":"d573c99e714bd1a59f9c6f4e9dfcb4564adf0c6febdcb569fd3a420ffcb8d53a"}
Jul 18 01:48:31.479863 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.479829    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88dq9" event={"ID":"d8131089-a28e-46d5-b800-de5c8e83457b","Type":"ContainerStarted","Data":"86f19c94756b874ec8fe1fba423d42a3b10797e3c9ab0e9faa86ec11f255ba38"}
Jul 18 01:48:31.494318 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.494202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" event={"ID":"934531a2-afe2-4bd6-ba63-3d716fbc8f15","Type":"ContainerStarted","Data":"5e9168fee1a205153c52c83fc0f39acc539893dff14a154ef89563aba962cd5a"}
Jul 18 01:48:31.503651 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.503547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6" event={"ID":"082d9fe7-b05b-448e-973d-a309eaac5b57","Type":"ContainerStarted","Data":"8501d721eb41bbb986c8b6ea515c38d00676a80f9d748ddbe7c86783b95be5aa"}
Jul 18 01:48:31.509987 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.509961    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lpmsm" event={"ID":"108086ce-a8d1-46d1-904d-86c21e823ad7","Type":"ContainerStarted","Data":"2096ca5af0380624145410e618d86aafa703e890a53cb793e424e0c5eab9b5e5"}
Jul 18 01:48:31.538720 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.538683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-drjsv" event={"ID":"9b0bef1c-ce07-4823-a070-285f60a843e5","Type":"ContainerStarted","Data":"61bbd072a7a940f5d25c3dd4c9c0d1741664a15caba697974524695b0a1d7fca"}
Jul 18 01:48:31.544074 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.544040    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jm445" event={"ID":"8222da46-c8c4-4d81-b240-a4455631215c","Type":"ContainerStarted","Data":"d090a4b0c0c1dc3cd63b118e68f19451d8935f3d0a5a612024668dcd077a5eea"}
Jul 18 01:48:31.562079 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.561791    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jk4xm" event={"ID":"8bc9c16a-612f-4d70-a673-79af0c13a275","Type":"ContainerStarted","Data":"683ff0b627011f7ad92b96ae8c7f1743cc2708ffd06147546d964a960ba0231b"}
Jul 18 01:48:31.581710 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:31.581672    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b9gn8" event={"ID":"3bc46aff-98b6-4d1d-81eb-eaa4022de5b2","Type":"ContainerStarted","Data":"af9a9cd805133bf0932aedb4fff5346921ffa7f59efdce0107053638f382b27d"}
Jul 18 01:48:32.595294 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:32.594798    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal" event={"ID":"9ee5ddcfe47f85210fa7b807fdc94725","Type":"ContainerStarted","Data":"cdc4b622a9976f618c2f05310d422b76e967cf9daf7833e8dd508476b3e8ec6d"}
Jul 18 01:48:32.612770 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:32.612703    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-56.ec2.internal" podStartSLOduration=3.612683455 podStartE2EDuration="3.612683455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:32.612375308 +0000 UTC m=+5.822089534" watchObservedRunningTime="2026-07-18 01:48:32.612683455 +0000 UTC m=+5.822397680"
Jul 18 01:48:32.943841 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:32.943730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:32.943841 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:32.943808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:32.944119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:32.943863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:32.944119 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:32.944004    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:32.944119 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:32.944077    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret podName:42e00b16-b4d4-4e92-8a85-b4477120c978 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:36.944054243 +0000 UTC m=+10.153768461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret") pod "global-pull-secret-syncer-qj8km" (UID: "42e00b16-b4d4-4e92-8a85-b4477120c978") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:32.944895 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:32.944590    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:32.944895 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:32.944654    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs podName:46d91c18-6dcf-4cb4-93a8-38d06290aa0a nodeName:}" failed. No retries permitted until 2026-07-18 01:48:36.944636549 +0000 UTC m=+10.154350754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs") pod "network-metrics-daemon-npj5j" (UID: "46d91c18-6dcf-4cb4-93a8-38d06290aa0a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:32.944895 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:32.944744    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:32.944895 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:32.944759    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:32.944895 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:32.944772    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tvshg for pod openshift-network-diagnostics/network-check-target-lxbgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:32.944895 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:32.944810    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg podName:56718ed9-ac2a-4497-8484-20b905ba2022 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:36.944797741 +0000 UTC m=+10.154511956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvshg" (UniqueName: "kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg") pod "network-check-target-lxbgv" (UID: "56718ed9-ac2a-4497-8484-20b905ba2022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:33.387872 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:33.387785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:33.388042 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:33.387940    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:33.388391 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:33.388368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:33.388510 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:33.388488    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:33.388588 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:33.388553    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:33.388645 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:33.388624    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:35.386187 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:35.385625    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:35.386187 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:35.385660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:35.386187 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:35.385630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:35.386187 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:35.385789    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:35.386187 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:35.385866    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:35.386187 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:35.385949    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:36.986070 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:36.986020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:36.986108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:36.986152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:36.986237    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:36.986255    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:36.986272    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:36.986282    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tvshg for pod openshift-network-diagnostics/network-check-target-lxbgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:36.986237    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:36.986325    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret podName:42e00b16-b4d4-4e92-8a85-b4477120c978 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:44.986301447 +0000 UTC m=+18.196015661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret") pod "global-pull-secret-syncer-qj8km" (UID: "42e00b16-b4d4-4e92-8a85-b4477120c978") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:36.986344    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs podName:46d91c18-6dcf-4cb4-93a8-38d06290aa0a nodeName:}" failed. No retries permitted until 2026-07-18 01:48:44.98633233 +0000 UTC m=+18.196046529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs") pod "network-metrics-daemon-npj5j" (UID: "46d91c18-6dcf-4cb4-93a8-38d06290aa0a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:36.986569 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:36.986356    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg podName:56718ed9-ac2a-4497-8484-20b905ba2022 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:44.986350186 +0000 UTC m=+18.196064385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvshg" (UniqueName: "kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg") pod "network-check-target-lxbgv" (UID: "56718ed9-ac2a-4497-8484-20b905ba2022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:37.386798 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:37.385987    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:37.386798 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:37.386098    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:37.386798 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:37.386484    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:37.386798 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:37.386585    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:37.386798 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:37.386644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:37.386798 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:37.386718    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:39.385926 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:39.385879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:39.386417 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:39.386024    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:39.386545 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:39.386506    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:39.386650 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:39.386627    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:39.386759 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:39.386743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:39.386881 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:39.386857    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:41.385543 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:41.385513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:41.385543 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:41.385552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:41.386119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:41.385552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:41.386119 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:41.385647    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:41.386119 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:41.385740    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:41.386258 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:41.386189    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:43.385412 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:43.385371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:43.385864 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:43.385372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:43.385864 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:43.385520    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:43.385864 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:43.385383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:43.385864 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:43.385581    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:43.385864 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:43.385679    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:45.049360 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:45.049319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:45.049368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:45.049414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.049482    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.049509    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.049548    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs podName:46d91c18-6dcf-4cb4-93a8-38d06290aa0a nodeName:}" failed. No retries permitted until 2026-07-18 01:49:01.049527681 +0000 UTC m=+34.259241895 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs") pod "network-metrics-daemon-npj5j" (UID: "46d91c18-6dcf-4cb4-93a8-38d06290aa0a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.049567    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret podName:42e00b16-b4d4-4e92-8a85-b4477120c978 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:01.049557888 +0000 UTC m=+34.259272085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret") pod "global-pull-secret-syncer-qj8km" (UID: "42e00b16-b4d4-4e92-8a85-b4477120c978") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.049566    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.049593    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.049607    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tvshg for pod openshift-network-diagnostics/network-check-target-lxbgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:45.049886 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.049654    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg podName:56718ed9-ac2a-4497-8484-20b905ba2022 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:01.04963691 +0000 UTC m=+34.259351123 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvshg" (UniqueName: "kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg") pod "network-check-target-lxbgv" (UID: "56718ed9-ac2a-4497-8484-20b905ba2022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:45.385314 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:45.385225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:45.385314 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:45.385248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:45.385314 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:45.385253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:45.385636 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.385346    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:45.385636 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.385416    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:45.385636 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:45.385497    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:47.386002 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:47.385964    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:47.386497 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:47.386113    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:47.386497 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:47.386488    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:47.386621 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:47.386584    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:47.386677 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:47.386653    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:47.386740 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:47.386722    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:49.384941 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:49.384900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:49.385505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:49.384976    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:49.385505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:49.385114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:49.385505 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:49.385127    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:49.385505 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:49.385219    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:49.385505 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:49.385295    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:50.635722 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.635293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pmjvt" event={"ID":"8dc3c51b-dafb-4e39-952e-ab0222934405","Type":"ContainerStarted","Data":"60302c78231b6437c168476b5df1a7d535c85a3887e6af12326cb32a9384a4cd"}
Jul 18 01:48:50.636982 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.636955    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8131089-a28e-46d5-b800-de5c8e83457b" containerID="1043ecbd109e131a7d5e387e3e10e99fa22d98a2875f0eb5d91e97d358daa0cb" exitCode=0
Jul 18 01:48:50.637099 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.637035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88dq9" event={"ID":"d8131089-a28e-46d5-b800-de5c8e83457b","Type":"ContainerDied","Data":"1043ecbd109e131a7d5e387e3e10e99fa22d98a2875f0eb5d91e97d358daa0cb"}
Jul 18 01:48:50.639315 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.639288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" event={"ID":"934531a2-afe2-4bd6-ba63-3d716fbc8f15","Type":"ContainerStarted","Data":"3eddd19d2c9f305f5d9c5bce9ee69eb7d066d66f4d0e745c2e2c6f210b3cd903"}
Jul 18 01:48:50.639415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.639368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" event={"ID":"934531a2-afe2-4bd6-ba63-3d716fbc8f15","Type":"ContainerStarted","Data":"93095fa16d6d26272fb5cc5355ab15917fa1c9e2208eeb028cbea09cd0726756"}
Jul 18 01:48:50.639476 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.639468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" event={"ID":"934531a2-afe2-4bd6-ba63-3d716fbc8f15","Type":"ContainerStarted","Data":"ea143708842141ac46935e6ad406b5ec7515442473a043d64028347bbf3acadd"}
Jul 18 01:48:50.640843 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.640822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6" event={"ID":"082d9fe7-b05b-448e-973d-a309eaac5b57","Type":"ContainerStarted","Data":"9701642dc586be8cf8fbcd02372f3519c88c2181d0d4afed03b066818d315add"}
Jul 18 01:48:50.642348 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.642327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lpmsm" event={"ID":"108086ce-a8d1-46d1-904d-86c21e823ad7","Type":"ContainerStarted","Data":"4a78ba938040b67140ea4967d1b0550c7344bbb4ca4ff5f7e4133bc8cc4cded1"}
Jul 18 01:48:50.643898 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.643874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jm445" event={"ID":"8222da46-c8c4-4d81-b240-a4455631215c","Type":"ContainerStarted","Data":"425524a3409b9f09df7735810ad66e74cfd793410890e80754f541fd5d2b59ab"}
Jul 18 01:48:50.645166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.645148    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jk4xm" event={"ID":"8bc9c16a-612f-4d70-a673-79af0c13a275","Type":"ContainerStarted","Data":"551e98c603cd187f29136cdc151d7243f24c7044898f79dc00debea4f13cb3b7"}
Jul 18 01:48:50.646365 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.646347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b9gn8" event={"ID":"3bc46aff-98b6-4d1d-81eb-eaa4022de5b2","Type":"ContainerStarted","Data":"fad38a9d6c6149ce35231432f51b4576525115b699c62b263e0cbc80baf58d23"}
Jul 18 01:48:50.652011 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.651972    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pmjvt" podStartSLOduration=2.8999016920000003 podStartE2EDuration="22.65196152s" totalImagesPullingTime="19.752059828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:50.651373222 +0000 UTC m=+23.861087442" watchObservedRunningTime="2026-07-18 01:48:50.65196152 +0000 UTC m=+23.861675805"
Jul 18 01:48:50.684818 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.684709    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b9gn8" podStartSLOduration=2.938571777 podStartE2EDuration="22.684693877s" totalImagesPullingTime="19.7461221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:50.683835321 +0000 UTC m=+23.893549541" watchObservedRunningTime="2026-07-18 01:48:50.684693877 +0000 UTC m=+23.894408127"
Jul 18 01:48:50.701032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.700992    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jm445" podStartSLOduration=2.896606835 podStartE2EDuration="22.700978012s" totalImagesPullingTime="19.804371177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:50.700475268 +0000 UTC m=+23.910189488" watchObservedRunningTime="2026-07-18 01:48:50.700978012 +0000 UTC m=+23.910692231"
Jul 18 01:48:50.716690 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.716648    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lpmsm" podStartSLOduration=2.965695057 podStartE2EDuration="22.716635146s" totalImagesPullingTime="19.750940089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:50.716600314 +0000 UTC m=+23.926314534" watchObservedRunningTime="2026-07-18 01:48:50.716635146 +0000 UTC m=+23.926349360"
Jul 18 01:48:50.737121 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.737083    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jk4xm" podStartSLOduration=2.993508733 podStartE2EDuration="22.737071014s" totalImagesPullingTime="19.743562281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:50.736787077 +0000 UTC m=+23.946501296" watchObservedRunningTime="2026-07-18 01:48:50.737071014 +0000 UTC m=+23.946785233"
Jul 18 01:48:50.783278 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.783252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:50.864506 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.864477    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:50.965112 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:50.965019    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:51.066209 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.066178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:51.121753 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.121724    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:51.385633 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.385533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:51.385802 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.385533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:51.385802 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:51.385672    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:51.385802 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.385533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:51.385802 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:51.385746    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:51.386026 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:51.385803    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:51.650258 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.650166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-drjsv" event={"ID":"9b0bef1c-ce07-4823-a070-285f60a843e5","Type":"ContainerStarted","Data":"b288b963e76c3940b7b571fcc11ac2723c8b821d07b4fdb41e8d413726978319"}
Jul 18 01:48:51.653534 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.653491    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" event={"ID":"934531a2-afe2-4bd6-ba63-3d716fbc8f15","Type":"ContainerStarted","Data":"2d5d34104a58f1c2a5ad095e20e22c9cb029878c31628b74caf83a5f985c9eab"}
Jul 18 01:48:51.653660 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.653572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" event={"ID":"934531a2-afe2-4bd6-ba63-3d716fbc8f15","Type":"ContainerStarted","Data":"3ff4ffdb2d1ca6b7e5e2dfd7cdd63834ef8232e77bf3d1b2fdb5931f1e63670b"}
Jul 18 01:48:51.653660 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.653624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" event={"ID":"934531a2-afe2-4bd6-ba63-3d716fbc8f15","Type":"ContainerStarted","Data":"7b185dda74f7a421cbba619a281e02272a3480dae31d396d4313f8d3dd48690d"}
Jul 18 01:48:51.668506 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.668460    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-drjsv" podStartSLOduration=3.923493293 podStartE2EDuration="23.668442564s" totalImagesPullingTime="19.744949271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:51.667727793 +0000 UTC m=+24.877442004" watchObservedRunningTime="2026-07-18 01:48:51.668442564 +0000 UTC m=+24.878156785"
Jul 18 01:48:51.795545 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.795509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:51.816517 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.816483    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 18 01:48:51.897020 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:51.896987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:52.261672 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:52.261538    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T01:48:51.816501737Z","UUID":"1fd9ee1c-de30-462d-b501-80f30b2cfbb9","Handler":null,"Name":"","Endpoint":""}
Jul 18 01:48:52.263197 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:52.263175    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 18 01:48:52.263363 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:52.263221    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 18 01:48:52.657432 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:52.657347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6" event={"ID":"082d9fe7-b05b-448e-973d-a309eaac5b57","Type":"ContainerStarted","Data":"40ee61c9fdb91c18215cf14fd2959654ced25d31211d3c6a34cd69d94a191bd2"}
Jul 18 01:48:53.017999 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.017971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:53.018534 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.018514    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:53.385273 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.385204    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:53.385533 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.385204    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:53.385533 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:53.385312    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:53.385533 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.385205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:53.385533 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:53.385384    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:53.385533 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:53.385492    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:53.663898 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.663820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" event={"ID":"934531a2-afe2-4bd6-ba63-3d716fbc8f15","Type":"ContainerStarted","Data":"9fce3637a88a131dafceccbda100b123e3e973d38e798d03bfde97ded04697be"}
Jul 18 01:48:53.666060 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.666021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6" event={"ID":"082d9fe7-b05b-448e-973d-a309eaac5b57","Type":"ContainerStarted","Data":"3aaeb57555ed066121d077e22dec6e61b6f6b7194f10f6334ea91be585d8ef3e"}
Jul 18 01:48:53.666217 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.666201    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:53.666800 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.666764    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b9gn8"
Jul 18 01:48:53.685752 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.685690    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hgn6" podStartSLOduration=3.726090924 podStartE2EDuration="25.685671378s" totalImagesPullingTime="21.959580454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:53.68516298 +0000 UTC m=+26.894877202" watchObservedRunningTime="2026-07-18 01:48:53.685671378 +0000 UTC m=+26.895385601"
Jul 18 01:48:53.750615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.750411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:53.945445 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:53.945409    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:54.005769 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:54.005737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:54.357834 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:54.357754    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:54.424914 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:54.424875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:55.385203 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:55.384948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:55.385809 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:55.385008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:55.385809 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:55.385249    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:55.385809 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:55.385062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:55.385809 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:55.385336    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:55.385809 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:55.385444    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:55.671603 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:55.671570    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8131089-a28e-46d5-b800-de5c8e83457b" containerID="580bb539f0d2d2bbb00299af636e1bff2a673af122b2ce81594b741d769e4221" exitCode=0
Jul 18 01:48:55.671751 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:55.671648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88dq9" event={"ID":"d8131089-a28e-46d5-b800-de5c8e83457b","Type":"ContainerDied","Data":"580bb539f0d2d2bbb00299af636e1bff2a673af122b2ce81594b741d769e4221"}
Jul 18 01:48:55.675370 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:55.675344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" event={"ID":"934531a2-afe2-4bd6-ba63-3d716fbc8f15","Type":"ContainerStarted","Data":"44c3f9ad0c86e61ac36763fbb7e89a89c3b1a960542d59687647f80bd3696534"}
Jul 18 01:48:55.719440 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:55.719396    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr" podStartSLOduration=7.918758202 podStartE2EDuration="27.719381947s" totalImagesPullingTime="19.800623745s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:55.719203294 +0000 UTC m=+28.928917514" watchObservedRunningTime="2026-07-18 01:48:55.719381947 +0000 UTC m=+28.929096166"
Jul 18 01:48:56.162215 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:56.162134    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:56.251901 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:56.251860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:56.678239 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:56.678208    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:56.678697 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:56.678338    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:56.678697 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:56.678387    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:56.690907 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:56.690883    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:56.693655 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:56.693633    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:56.694533 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:56.694508    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:48:56.751957 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:56.751922    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:57.146878 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.146845    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qj8km"]
Jul 18 01:48:57.147161 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.146964    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:57.147161 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:57.147077    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:57.150113 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.150085    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-npj5j"]
Jul 18 01:48:57.150238 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.150212    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:57.150357 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:57.150329    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:57.150768 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.150750    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lxbgv"]
Jul 18 01:48:57.150856 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.150845    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:57.150937 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:57.150922    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:57.162479 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.162453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:57.260528 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.260449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:57.450973 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.450942    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:57.549691 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.549601    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:57.644269 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.644236    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:57.682143 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.682112    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8131089-a28e-46d5-b800-de5c8e83457b" containerID="61cbda02fb37b3e87f6ed78e181fa881ec5bb8cb775bfa2c5c57ee00314b549f" exitCode=0
Jul 18 01:48:57.682575 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:57.682193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88dq9" event={"ID":"d8131089-a28e-46d5-b800-de5c8e83457b","Type":"ContainerDied","Data":"61cbda02fb37b3e87f6ed78e181fa881ec5bb8cb775bfa2c5c57ee00314b549f"}
Jul 18 01:48:58.385538 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:58.385507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:48:58.385697 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:58.385609    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:48:58.443078 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:58.443030    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jk4xm_8bc9c16a-612f-4d70-a673-79af0c13a275/dns-node-resolver/0.log"
Jul 18 01:48:59.060265 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:59.060025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:59.120812 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:59.120782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:59.385550 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:59.385476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:48:59.385696 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:59.385589    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:48:59.385696 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:59.385617    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:48:59.385696 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:48:59.385681    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:48:59.442333 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:59.442305    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lpmsm_108086ce-a8d1-46d1-904d-86c21e823ad7/node-ca/0.log"
Jul 18 01:48:59.688848 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:59.688816    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8131089-a28e-46d5-b800-de5c8e83457b" containerID="61d59ff94119a8e8fe8e958850777d5c96f607ac24d0389cbf69aea708184754" exitCode=0
Jul 18 01:48:59.688997 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:59.688880    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88dq9" event={"ID":"d8131089-a28e-46d5-b800-de5c8e83457b","Type":"ContainerDied","Data":"61d59ff94119a8e8fe8e958850777d5c96f607ac24d0389cbf69aea708184754"}
Jul 18 01:48:59.985024 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:48:59.984944    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:00.385278 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:00.385191    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:00.385919 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:00.385316    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:01.074973 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:01.074937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:01.075184 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:01.075000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:01.075184 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:01.075060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:01.075184 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.075139    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:49:01.075184 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.075160    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:49:01.075368 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.075144    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:49:01.075368 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.075215    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret podName:42e00b16-b4d4-4e92-8a85-b4477120c978 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:33.075193686 +0000 UTC m=+66.284907894 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret") pod "global-pull-secret-syncer-qj8km" (UID: "42e00b16-b4d4-4e92-8a85-b4477120c978") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:49:01.075368 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.075223    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:49:01.075368 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.075237    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tvshg for pod openshift-network-diagnostics/network-check-target-lxbgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:49:01.075368 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.075238    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs podName:46d91c18-6dcf-4cb4-93a8-38d06290aa0a nodeName:}" failed. No retries permitted until 2026-07-18 01:49:33.075227048 +0000 UTC m=+66.284941253 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs") pod "network-metrics-daemon-npj5j" (UID: "46d91c18-6dcf-4cb4-93a8-38d06290aa0a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:49:01.075368 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.075273    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg podName:56718ed9-ac2a-4497-8484-20b905ba2022 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:33.075258184 +0000 UTC m=+66.284972397 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvshg" (UniqueName: "kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg") pod "network-check-target-lxbgv" (UID: "56718ed9-ac2a-4497-8484-20b905ba2022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:49:01.384623 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:01.384554    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:01.384769 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.384657    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:01.384825 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:01.384784    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:01.384903 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:01.384882    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:01.427995 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:01.427961    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:02.385320 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:02.385284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:02.385505 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:02.385417    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:02.954396 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:02.954358    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:03.055966 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:03.055938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:03.128282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:03.128245    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:03.384929 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:03.384842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:03.385123 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:03.384989    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:03.385426 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:03.385407    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:03.385561 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:03.385533    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:03.468234 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:03.468198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:03.566724 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:03.566691    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:03.667838 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:03.667811    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:03.870881 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:03.870849    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:04.107231 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:04.107202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:04.384706 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:04.384627    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:04.384836 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:04.384730    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:04.662744 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:04.662659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:04.704804 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:04.704774    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8131089-a28e-46d5-b800-de5c8e83457b" containerID="e4cf60b26e82618b8ab9d219bf9f6de9643e94189a3a39a0820818a7dbd66eb5" exitCode=0
Jul 18 01:49:04.704942 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:04.704836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88dq9" event={"ID":"d8131089-a28e-46d5-b800-de5c8e83457b","Type":"ContainerDied","Data":"e4cf60b26e82618b8ab9d219bf9f6de9643e94189a3a39a0820818a7dbd66eb5"}
Jul 18 01:49:05.055008 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:05.054977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:05.384907 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:05.384821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:05.384907 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:05.384855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:05.385376 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:05.384948    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:05.385376 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:05.385082    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:05.693649 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:05.693571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:05.710264 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:05.710239    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8131089-a28e-46d5-b800-de5c8e83457b" containerID="5852f217e175da17aa9498399b56a2fb917f92db15e0fc3386f3ab655e4a2ded" exitCode=0
Jul 18 01:49:05.710407 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:05.710291    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88dq9" event={"ID":"d8131089-a28e-46d5-b800-de5c8e83457b","Type":"ContainerDied","Data":"5852f217e175da17aa9498399b56a2fb917f92db15e0fc3386f3ab655e4a2ded"}
Jul 18 01:49:06.100010 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:06.099744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:06.220671 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:06.220631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:06.293527 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:06.293496    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:06.385154 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:06.385093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:06.385480 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:06.385193    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:06.454673 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:06.454646    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:06.715752 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:06.715723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88dq9" event={"ID":"d8131089-a28e-46d5-b800-de5c8e83457b","Type":"ContainerStarted","Data":"04f5dd3de47b9fe52da545be84c881da13793395c9fa0ab016f6b76b42a11fcc"}
Jul 18 01:49:06.750508 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:06.750459    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-88dq9" podStartSLOduration=8.279423314 podStartE2EDuration="38.750443497s" totalImagesPullingTime="30.471020183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:06.750100259 +0000 UTC m=+39.959814471" watchObservedRunningTime="2026-07-18 01:49:06.750443497 +0000 UTC m=+39.960157717"
Jul 18 01:49:06.866330 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:06.866297    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:06.936256 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:06.936225    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:07.052497 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:07.052412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:07.387842 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:07.387761    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:07.388204 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:07.387761    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:07.388204 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:07.387862    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:07.388204 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:07.387943    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:07.449638 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:07.449609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:07.514015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:07.513981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:08.051411 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:08.051378    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:08.384894 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:08.384818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:08.385028 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:08.384917    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:08.448217 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:08.448191    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:08.848057 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:08.848005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:09.055837 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:09.055810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:09.119965 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:09.119895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:09.248756 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:09.248722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:09.385162 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:09.385085    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:09.385305 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:09.385098    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:09.385305 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:09.385201    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:09.385434 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:09.385299    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:10.250697 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:10.250662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:10.385206 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:10.385179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:10.385363 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:10.385283    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:10.642099 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:10.641998    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:10.885004 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:10.884975    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:11.045976 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:11.045947    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:11.248940 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:11.248905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:11.385556 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:11.385486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:11.385865 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:11.385609    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:11.385865 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:11.385658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:11.385865 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:11.385762    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:11.549378 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:11.549346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:12.385282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:12.385255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:12.385472 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:12.385369    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:12.471880 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:12.471852    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:12.643022 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:12.642946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:12.841560 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:12.841530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:13.041643 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:13.041609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:13.242875 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:13.242842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:13.385260 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:13.385180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:13.385260 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:13.385228    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:13.385478 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:13.385301    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:13.385478 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:13.385428    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:13.650493 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:13.650409    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:14.385176 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:14.385150    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:14.385362 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:14.385264    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:15.384803 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:15.384774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:15.385336 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:15.384788    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:15.385336 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:15.384902    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:15.385336 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:15.384959    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:16.384573 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:16.384538    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:16.384747 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:16.384642    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:17.386527 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:17.386491    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:17.386985 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:17.386619    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:17.386985 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:17.386677    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:17.386985 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:17.386795    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:18.385055 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:18.385010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:18.385216 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:18.385120    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qj8km" podUID="42e00b16-b4d4-4e92-8a85-b4477120c978"
Jul 18 01:49:19.285122 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:19.285086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:19.385526 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:19.385493    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:19.385795 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:19.385632    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npj5j" podUID="46d91c18-6dcf-4cb4-93a8-38d06290aa0a"
Jul 18 01:49:19.385795 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:19.385676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:19.385795 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:19.385759    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxbgv" podUID="56718ed9-ac2a-4497-8484-20b905ba2022"
Jul 18 01:49:19.534946 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:19.534913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:19.636263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:19.636179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:20.153170 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.153142    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-56.ec2.internal" event="NodeReady"
Jul 18 01:49:20.153323 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.153262    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 18 01:49:20.200943 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.200909    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xz6vx"]
Jul 18 01:49:20.222178 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.222153    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qgmwb"]
Jul 18 01:49:20.222330 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.222249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xz6vx"
Jul 18 01:49:20.225071 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.225026    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s99dg\""
Jul 18 01:49:20.225368 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.225343    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 18 01:49:20.225468 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.225435    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 18 01:49:20.225528 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.225435    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 18 01:49:20.242079 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.242056    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xz6vx"]
Jul 18 01:49:20.242169 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.242085    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qgmwb"]
Jul 18 01:49:20.242169 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.242163    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.244954 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.244937    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 18 01:49:20.245130 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.244996    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 18 01:49:20.245302 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.245257    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 18 01:49:20.245302 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.245297    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 18 01:49:20.245490 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.245477    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6l5cz\""
Jul 18 01:49:20.306018 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.305994    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xrhq8"]
Jul 18 01:49:20.312769 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.312744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dad73e0c-2967-4b12-8d9f-c45337cedb3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.312834 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.312778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gg57\" (UniqueName: \"kubernetes.io/projected/dad73e0c-2967-4b12-8d9f-c45337cedb3f-kube-api-access-2gg57\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.312834 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.312811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dad73e0c-2967-4b12-8d9f-c45337cedb3f-crio-socket\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.312899 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.312877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dad73e0c-2967-4b12-8d9f-c45337cedb3f-data-volume\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.312932 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.312902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8m9c\" (UniqueName: \"kubernetes.io/projected/892559b2-c9ea-43d1-8f23-0e41d98f63a3-kube-api-access-l8m9c\") pod \"ingress-canary-xz6vx\" (UID: \"892559b2-c9ea-43d1-8f23-0e41d98f63a3\") " pod="openshift-ingress-canary/ingress-canary-xz6vx"
Jul 18 01:49:20.312967 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.312956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dad73e0c-2967-4b12-8d9f-c45337cedb3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.313007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.312994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/892559b2-c9ea-43d1-8f23-0e41d98f63a3-cert\") pod \"ingress-canary-xz6vx\" (UID: \"892559b2-c9ea-43d1-8f23-0e41d98f63a3\") " pod="openshift-ingress-canary/ingress-canary-xz6vx"
Jul 18 01:49:20.324220 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.324203    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.328012 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.327991    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zdfxb\""
Jul 18 01:49:20.328160 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.328080    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 18 01:49:20.328160 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.328089    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 18 01:49:20.329387 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.329366    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xrhq8"]
Jul 18 01:49:20.385186 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.385164    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:20.387872 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.387854    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:49:20.414134 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dad73e0c-2967-4b12-8d9f-c45337cedb3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.414134 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acfc916f-b1f4-4271-a49e-b20783422922-metrics-tls\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.414252 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/892559b2-c9ea-43d1-8f23-0e41d98f63a3-cert\") pod \"ingress-canary-xz6vx\" (UID: \"892559b2-c9ea-43d1-8f23-0e41d98f63a3\") " pod="openshift-ingress-canary/ingress-canary-xz6vx"
Jul 18 01:49:20.414252 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dad73e0c-2967-4b12-8d9f-c45337cedb3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.414343 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gg57\" (UniqueName: \"kubernetes.io/projected/dad73e0c-2967-4b12-8d9f-c45337cedb3f-kube-api-access-2gg57\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.414385 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dad73e0c-2967-4b12-8d9f-c45337cedb3f-crio-socket\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.414435 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acfc916f-b1f4-4271-a49e-b20783422922-tmp-dir\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.414435 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dad73e0c-2967-4b12-8d9f-c45337cedb3f-data-volume\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.414544 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8m9c\" (UniqueName: \"kubernetes.io/projected/892559b2-c9ea-43d1-8f23-0e41d98f63a3-kube-api-access-l8m9c\") pod \"ingress-canary-xz6vx\" (UID: \"892559b2-c9ea-43d1-8f23-0e41d98f63a3\") " pod="openshift-ingress-canary/ingress-canary-xz6vx"
Jul 18 01:49:20.414544 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acfc916f-b1f4-4271-a49e-b20783422922-config-volume\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.414544 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4x47\" (UniqueName: \"kubernetes.io/projected/acfc916f-b1f4-4271-a49e-b20783422922-kube-api-access-b4x47\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.414708 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414561    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dad73e0c-2967-4b12-8d9f-c45337cedb3f-crio-socket\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.414708 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dad73e0c-2967-4b12-8d9f-c45337cedb3f-data-volume\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.414786 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.414754    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dad73e0c-2967-4b12-8d9f-c45337cedb3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.418204 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.418187    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dad73e0c-2967-4b12-8d9f-c45337cedb3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.418270 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.418221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/892559b2-c9ea-43d1-8f23-0e41d98f63a3-cert\") pod \"ingress-canary-xz6vx\" (UID: \"892559b2-c9ea-43d1-8f23-0e41d98f63a3\") " pod="openshift-ingress-canary/ingress-canary-xz6vx"
Jul 18 01:49:20.423368 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.423345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gg57\" (UniqueName: \"kubernetes.io/projected/dad73e0c-2967-4b12-8d9f-c45337cedb3f-kube-api-access-2gg57\") pod \"insights-runtime-extractor-qgmwb\" (UID: \"dad73e0c-2967-4b12-8d9f-c45337cedb3f\") " pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.423511 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.423494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8m9c\" (UniqueName: \"kubernetes.io/projected/892559b2-c9ea-43d1-8f23-0e41d98f63a3-kube-api-access-l8m9c\") pod \"ingress-canary-xz6vx\" (UID: \"892559b2-c9ea-43d1-8f23-0e41d98f63a3\") " pod="openshift-ingress-canary/ingress-canary-xz6vx"
Jul 18 01:49:20.434946 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.434919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:20.500846 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.500818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:20.515094 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.515075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acfc916f-b1f4-4271-a49e-b20783422922-metrics-tls\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.515155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.515131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acfc916f-b1f4-4271-a49e-b20783422922-tmp-dir\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.515155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.515152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acfc916f-b1f4-4271-a49e-b20783422922-config-volume\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.515219 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.515168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4x47\" (UniqueName: \"kubernetes.io/projected/acfc916f-b1f4-4271-a49e-b20783422922-kube-api-access-b4x47\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.515495 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.515476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acfc916f-b1f4-4271-a49e-b20783422922-tmp-dir\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.515756 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.515734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acfc916f-b1f4-4271-a49e-b20783422922-config-volume\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.517260 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.517242    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acfc916f-b1f4-4271-a49e-b20783422922-metrics-tls\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.522621 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.522602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4x47\" (UniqueName: \"kubernetes.io/projected/acfc916f-b1f4-4271-a49e-b20783422922-kube-api-access-b4x47\") pod \"dns-default-xrhq8\" (UID: \"acfc916f-b1f4-4271-a49e-b20783422922\") " pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.531542 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.531523    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xz6vx"
Jul 18 01:49:20.549712 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.549693    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qgmwb"
Jul 18 01:49:20.632888 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.632847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:20.747527 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.747013    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qgmwb"]
Jul 18 01:49:20.748150 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.747890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xz6vx"]
Jul 18 01:49:20.749997 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:20.749966    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad73e0c_2967_4b12_8d9f_c45337cedb3f.slice/crio-5c03974556a35ac6fc578312504a1c94b99d3510a511c878f09caf0a70caceae WatchSource:0}: Error finding container 5c03974556a35ac6fc578312504a1c94b99d3510a511c878f09caf0a70caceae: Status 404 returned error can't find the container with id 5c03974556a35ac6fc578312504a1c94b99d3510a511c878f09caf0a70caceae
Jul 18 01:49:20.750480 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:20.750453    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892559b2_c9ea_43d1_8f23_0e41d98f63a3.slice/crio-6c5c699f7e28cebb1c0775da9cd4d15430d7e313f3dab8c972cec8c99ded15ca WatchSource:0}: Error finding container 6c5c699f7e28cebb1c0775da9cd4d15430d7e313f3dab8c972cec8c99ded15ca: Status 404 returned error can't find the container with id 6c5c699f7e28cebb1c0775da9cd4d15430d7e313f3dab8c972cec8c99ded15ca
Jul 18 01:49:20.793579 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:20.793551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xrhq8"]
Jul 18 01:49:20.796593 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:20.796563    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacfc916f_b1f4_4271_a49e_b20783422922.slice/crio-99b2129993d323abf12c2be3df7236f4950a95d22881b559e8b6a058ea9f55b1 WatchSource:0}: Error finding container 99b2129993d323abf12c2be3df7236f4950a95d22881b559e8b6a058ea9f55b1: Status 404 returned error can't find the container with id 99b2129993d323abf12c2be3df7236f4950a95d22881b559e8b6a058ea9f55b1
Jul 18 01:49:21.026577 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.026548    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:21.114915 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.114876    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:21.385770 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.384684    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:21.385770 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.385395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:21.389470 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.389140    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4m6k\""
Jul 18 01:49:21.389470 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.389191    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gzg6q\""
Jul 18 01:49:21.389470 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.389140    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:49:21.389470 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.389411    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 01:49:21.389746 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.389534    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 01:49:21.754329 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.754291    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xrhq8" event={"ID":"acfc916f-b1f4-4271-a49e-b20783422922","Type":"ContainerStarted","Data":"99b2129993d323abf12c2be3df7236f4950a95d22881b559e8b6a058ea9f55b1"}
Jul 18 01:49:21.756964 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.756937    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qgmwb" event={"ID":"dad73e0c-2967-4b12-8d9f-c45337cedb3f","Type":"ContainerStarted","Data":"5c03974556a35ac6fc578312504a1c94b99d3510a511c878f09caf0a70caceae"}
Jul 18 01:49:21.758654 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:21.758627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xz6vx" event={"ID":"892559b2-c9ea-43d1-8f23-0e41d98f63a3","Type":"ContainerStarted","Data":"6c5c699f7e28cebb1c0775da9cd4d15430d7e313f3dab8c972cec8c99ded15ca"}
Jul 18 01:49:24.486295 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:24.486261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:24.584061 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:24.584015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:26.307782 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.307746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:26.411648 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.411562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:26.490107 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.490078    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"]
Jul 18 01:49:26.490405 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.490385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:26.500182 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.500159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"
Jul 18 01:49:26.502938 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.502918    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 18 01:49:26.502938 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.502934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-fz487\""
Jul 18 01:49:26.506808 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.506463    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"]
Jul 18 01:49:26.564838 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.564770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7545c793-d408-4399-bd1b-adcf7ceddc7f-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dcxqz\" (UID: \"7545c793-d408-4399-bd1b-adcf7ceddc7f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"
Jul 18 01:49:26.657310 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.657269    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:26.665283 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.665256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7545c793-d408-4399-bd1b-adcf7ceddc7f-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dcxqz\" (UID: \"7545c793-d408-4399-bd1b-adcf7ceddc7f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"
Jul 18 01:49:26.668140 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.668112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7545c793-d408-4399-bd1b-adcf7ceddc7f-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dcxqz\" (UID: \"7545c793-d408-4399-bd1b-adcf7ceddc7f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"
Jul 18 01:49:26.809254 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:26.809213    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"
Jul 18 01:49:27.353089 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:27.353062    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"]
Jul 18 01:49:27.358301 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:27.358217    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7545c793_d408_4399_bd1b_adcf7ceddc7f.slice/crio-e5493b415578d1798af5e437eb5b43b1e34247024be53a3ed7acadc90c13a644 WatchSource:0}: Error finding container e5493b415578d1798af5e437eb5b43b1e34247024be53a3ed7acadc90c13a644: Status 404 returned error can't find the container with id e5493b415578d1798af5e437eb5b43b1e34247024be53a3ed7acadc90c13a644
Jul 18 01:49:27.785683 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:27.785643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xrhq8" event={"ID":"acfc916f-b1f4-4271-a49e-b20783422922","Type":"ContainerStarted","Data":"39396a87d163c74b3c1087aa7782979639dada667c8d935d908503c98854b992"}
Jul 18 01:49:27.785867 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:27.785746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xrhq8" event={"ID":"acfc916f-b1f4-4271-a49e-b20783422922","Type":"ContainerStarted","Data":"653f677a9533423aab6de39f9deb5a95913740dfcaa4a25ec5bc9baee89a0ddb"}
Jul 18 01:49:27.786183 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:27.786163    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:27.790016 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:27.789940    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qgmwb" event={"ID":"dad73e0c-2967-4b12-8d9f-c45337cedb3f","Type":"ContainerStarted","Data":"cbd9a657a58700b4163fe9cc4ffe73e23416b524a3714e46457062bc4dcce215"}
Jul 18 01:49:27.792970 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:27.792946    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xz6vx" event={"ID":"892559b2-c9ea-43d1-8f23-0e41d98f63a3","Type":"ContainerStarted","Data":"c8a5eac9cd748d7fc3ef7db2514d00542eb69f9e25c8137c28bc6b4f67f1a7bf"}
Jul 18 01:49:27.794873 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:27.794850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz" event={"ID":"7545c793-d408-4399-bd1b-adcf7ceddc7f","Type":"ContainerStarted","Data":"e5493b415578d1798af5e437eb5b43b1e34247024be53a3ed7acadc90c13a644"}
Jul 18 01:49:27.805013 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:27.804958    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xrhq8" podStartSLOduration=1.3947493469999999 podStartE2EDuration="7.804941055s" totalImagesPullingTime="6.410191708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:27.803954529 +0000 UTC m=+61.013668752" watchObservedRunningTime="2026-07-18 01:49:27.804941055 +0000 UTC m=+61.014655275"
Jul 18 01:49:27.820918 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:27.820869    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xz6vx" podStartSLOduration=1.364977048 podStartE2EDuration="7.820850654s" totalImagesPullingTime="6.455873606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:27.819572946 +0000 UTC m=+61.029287168" watchObservedRunningTime="2026-07-18 01:49:27.820850654 +0000 UTC m=+61.030564879"
Jul 18 01:49:28.697599 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:28.697517    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmfpr"
Jul 18 01:49:28.799927 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:28.799887    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz" event={"ID":"7545c793-d408-4399-bd1b-adcf7ceddc7f","Type":"ContainerStarted","Data":"31f25ff3b9096e66a46016cfd022de829655697a0bf711fdb07dce1f8a6c2e84"}
Jul 18 01:49:28.800379 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:28.800355    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"
Jul 18 01:49:28.805701 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:28.805678    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz"
Jul 18 01:49:28.819381 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:28.819322    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dcxqz" podStartSLOduration=1.759492109 podStartE2EDuration="2.819301574s" totalImagesPullingTime="1.059809465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:28.818667582 +0000 UTC m=+62.028381802" watchObservedRunningTime="2026-07-18 01:49:28.819301574 +0000 UTC m=+62.029015794"
Jul 18 01:49:28.828728 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:28.828703    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:28.925284 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:28.925247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:29.508287 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.508257    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:29.577689 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.577658    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"]
Jul 18 01:49:29.588521 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.588495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.591277 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.591251    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 18 01:49:29.592573 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.592547    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"]
Jul 18 01:49:29.592794 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.592773    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 18 01:49:29.592899 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.592865    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 18 01:49:29.593018 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.592997    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2gz54\""
Jul 18 01:49:29.594084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.594001    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 18 01:49:29.594355 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.594328    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 18 01:49:29.598620 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.598590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:29.692630 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.692589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/719da8da-43ff-4692-ad70-b84987e714ce-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.692630 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.692629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/719da8da-43ff-4692-ad70-b84987e714ce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.692881 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.692666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/719da8da-43ff-4692-ad70-b84987e714ce-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.692881 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.692777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpgqp\" (UniqueName: \"kubernetes.io/projected/719da8da-43ff-4692-ad70-b84987e714ce-kube-api-access-hpgqp\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.793589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.793499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/719da8da-43ff-4692-ad70-b84987e714ce-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.794034 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.793588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpgqp\" (UniqueName: \"kubernetes.io/projected/719da8da-43ff-4692-ad70-b84987e714ce-kube-api-access-hpgqp\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.794034 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.793682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/719da8da-43ff-4692-ad70-b84987e714ce-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.794034 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.793712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/719da8da-43ff-4692-ad70-b84987e714ce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.794034 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:29.793890    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 18 01:49:29.794034 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:29.793971    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719da8da-43ff-4692-ad70-b84987e714ce-prometheus-operator-tls podName:719da8da-43ff-4692-ad70-b84987e714ce nodeName:}" failed. No retries permitted until 2026-07-18 01:49:30.293949712 +0000 UTC m=+63.503663929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/719da8da-43ff-4692-ad70-b84987e714ce-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-vf87r" (UID: "719da8da-43ff-4692-ad70-b84987e714ce") : secret "prometheus-operator-tls" not found
Jul 18 01:49:29.794553 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.794523    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/719da8da-43ff-4692-ad70-b84987e714ce-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.796748 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.796724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/719da8da-43ff-4692-ad70-b84987e714ce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:29.805524 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:29.805503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpgqp\" (UniqueName: \"kubernetes.io/projected/719da8da-43ff-4692-ad70-b84987e714ce-kube-api-access-hpgqp\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:30.107904 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:30.107830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:30.296972 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:30.296937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/719da8da-43ff-4692-ad70-b84987e714ce-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:30.299694 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:30.299666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/719da8da-43ff-4692-ad70-b84987e714ce-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vf87r\" (UID: \"719da8da-43ff-4692-ad70-b84987e714ce\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:30.501276 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:30.501237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"
Jul 18 01:49:30.836196 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:30.836166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-vf87r"]
Jul 18 01:49:30.839373 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:30.839345    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719da8da_43ff_4692_ad70_b84987e714ce.slice/crio-18a5f7868affdfd703287192ca2f6ae2b2f35e40ab6e5ae8d7adb835ed0138e9 WatchSource:0}: Error finding container 18a5f7868affdfd703287192ca2f6ae2b2f35e40ab6e5ae8d7adb835ed0138e9: Status 404 returned error can't find the container with id 18a5f7868affdfd703287192ca2f6ae2b2f35e40ab6e5ae8d7adb835ed0138e9
Jul 18 01:49:31.813227 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:31.813186    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qgmwb" event={"ID":"dad73e0c-2967-4b12-8d9f-c45337cedb3f","Type":"ContainerStarted","Data":"545d57fb55116812089085337b1b3db3a069c766f78d950a3d45ef0a93cec0f2"}
Jul 18 01:49:31.814531 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:31.814501    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r" event={"ID":"719da8da-43ff-4692-ad70-b84987e714ce","Type":"ContainerStarted","Data":"18a5f7868affdfd703287192ca2f6ae2b2f35e40ab6e5ae8d7adb835ed0138e9"}
Jul 18 01:49:32.306617 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:32.306583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:32.820390 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:32.820352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r" event={"ID":"719da8da-43ff-4692-ad70-b84987e714ce","Type":"ContainerStarted","Data":"1d907d1d434628260a66a3bd21b3c32d306ffe1c86e32c31bcd4d15ed801cdc2"}
Jul 18 01:49:32.820586 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:32.820417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r" event={"ID":"719da8da-43ff-4692-ad70-b84987e714ce","Type":"ContainerStarted","Data":"4198c59027692dc8f4896acbeece8a71054de6ee3f6150be2140b0270edf36c8"}
Jul 18 01:49:32.839650 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:32.839601    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vf87r" podStartSLOduration=2.692106118 podStartE2EDuration="3.839586958s" totalImagesPullingTime="1.14748084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:32.838850243 +0000 UTC m=+66.048564464" watchObservedRunningTime="2026-07-18 01:49:32.839586958 +0000 UTC m=+66.049301178"
Jul 18 01:49:33.119957 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.119862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:33.119957 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.119928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:33.120187 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.119982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:33.123101 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.123079    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 01:49:33.123220 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.123179    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:49:33.124176 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.124161    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:49:33.133313 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.133292    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 01:49:33.133497 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.133481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46d91c18-6dcf-4cb4-93a8-38d06290aa0a-metrics-certs\") pod \"network-metrics-daemon-npj5j\" (UID: \"46d91c18-6dcf-4cb4-93a8-38d06290aa0a\") " pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:33.133557 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.133546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42e00b16-b4d4-4e92-8a85-b4477120c978-original-pull-secret\") pod \"global-pull-secret-syncer-qj8km\" (UID: \"42e00b16-b4d4-4e92-8a85-b4477120c978\") " pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:33.144175 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.144153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvshg\" (UniqueName: \"kubernetes.io/projected/56718ed9-ac2a-4497-8484-20b905ba2022-kube-api-access-tvshg\") pod \"network-check-target-lxbgv\" (UID: \"56718ed9-ac2a-4497-8484-20b905ba2022\") " pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:33.293180 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.293147    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qj8km"
Jul 18 01:49:33.404148 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.404070    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4m6k\""
Jul 18 01:49:33.409999 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.409973    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gzg6q\""
Jul 18 01:49:33.411905 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.411885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npj5j"
Jul 18 01:49:33.417829 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:33.417811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:34.453248 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.453195    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-npj5j"]
Jul 18 01:49:34.457631 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:34.457599    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46d91c18_6dcf_4cb4_93a8_38d06290aa0a.slice/crio-ca0c86556b0269cea64b5b60f5961d681158a0586679aff413bbde40c620aadf WatchSource:0}: Error finding container ca0c86556b0269cea64b5b60f5961d681158a0586679aff413bbde40c620aadf: Status 404 returned error can't find the container with id ca0c86556b0269cea64b5b60f5961d681158a0586679aff413bbde40c620aadf
Jul 18 01:49:34.466621 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.466594    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qj8km"]
Jul 18 01:49:34.469315 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:34.469292    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e00b16_b4d4_4e92_8a85_b4477120c978.slice/crio-cc4aaa23538be9d0fc49790584914ec83c90a8db192c7fb4bc6015966c6808dd WatchSource:0}: Error finding container cc4aaa23538be9d0fc49790584914ec83c90a8db192c7fb4bc6015966c6808dd: Status 404 returned error can't find the container with id cc4aaa23538be9d0fc49790584914ec83c90a8db192c7fb4bc6015966c6808dd
Jul 18 01:49:34.506622 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.506600    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lxbgv"]
Jul 18 01:49:34.509546 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:34.509522    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56718ed9_ac2a_4497_8484_20b905ba2022.slice/crio-3902986fe33b06b04c2cfe95ff2de33f681a306d90cf21f7abf5f7baa4326c48 WatchSource:0}: Error finding container 3902986fe33b06b04c2cfe95ff2de33f681a306d90cf21f7abf5f7baa4326c48: Status 404 returned error can't find the container with id 3902986fe33b06b04c2cfe95ff2de33f681a306d90cf21f7abf5f7baa4326c48
Jul 18 01:49:34.669033 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.668996    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:34.767755 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.767727    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:34.829662 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.829629    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qgmwb" event={"ID":"dad73e0c-2967-4b12-8d9f-c45337cedb3f","Type":"ContainerStarted","Data":"8d5c32c70b3ff2f2cd5dae4b820caf6f1552d01b4945d7649f709bca5a989eaa"}
Jul 18 01:49:34.830849 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.830817    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lxbgv" event={"ID":"56718ed9-ac2a-4497-8484-20b905ba2022","Type":"ContainerStarted","Data":"3902986fe33b06b04c2cfe95ff2de33f681a306d90cf21f7abf5f7baa4326c48"}
Jul 18 01:49:34.832031 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.832012    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qj8km" event={"ID":"42e00b16-b4d4-4e92-8a85-b4477120c978","Type":"ContainerStarted","Data":"cc4aaa23538be9d0fc49790584914ec83c90a8db192c7fb4bc6015966c6808dd"}
Jul 18 01:49:34.833094 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.833070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-npj5j" event={"ID":"46d91c18-6dcf-4cb4-93a8-38d06290aa0a","Type":"ContainerStarted","Data":"ca0c86556b0269cea64b5b60f5961d681158a0586679aff413bbde40c620aadf"}
Jul 18 01:49:34.853074 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.853009    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qgmwb" podStartSLOduration=1.446963021 podStartE2EDuration="14.852998435s" totalImagesPullingTime="13.406035414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:49:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:34.852473657 +0000 UTC m=+68.062187877" watchObservedRunningTime="2026-07-18 01:49:34.852998435 +0000 UTC m=+68.062712655"
Jul 18 01:49:34.878367 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.878338    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:34.925012 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.924940    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-vhx57"]
Jul 18 01:49:34.927520 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.927479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:34.930694 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.930639    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 18 01:49:34.930694 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.930688    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tx5vf\""
Jul 18 01:49:34.931262 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.931243    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 18 01:49:34.931343 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.931283    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 18 01:49:34.943683 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.943655    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-vhx57"]
Jul 18 01:49:34.957544 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.957519    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rg7pk"]
Jul 18 01:49:34.960223 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.960201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:34.963589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.963570    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6ncsh\""
Jul 18 01:49:34.963589 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.963581    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 18 01:49:34.963747 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.963589    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 18 01:49:34.963885 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.963869    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 18 01:49:34.975509 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:34.975436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84zn7\" (UniqueName: \"kubernetes.io/projected/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-api-access-84zn7\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-sys\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-root\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-wtmp\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgwk2\" (UniqueName: \"kubernetes.io/projected/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-kube-api-access-lgwk2\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-tls\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-metrics-client-ca\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.037032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.036845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-textfile\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.075116 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.075075    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.138020 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.137983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-wtmp\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.138282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgwk2\" (UniqueName: \"kubernetes.io/projected/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-kube-api-access-lgwk2\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.138282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-tls\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-metrics-client-ca\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.138282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-textfile\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.138282 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84zn7\" (UniqueName: \"kubernetes.io/projected/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-api-access-84zn7\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.138792 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138309    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138792 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-sys\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138792 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138792 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.138792 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-root\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138792 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-root\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.138792 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.138669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-wtmp\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.139336 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.139310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-textfile\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.139602 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.139580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.140096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.140071    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.140217 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.140132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-metrics-client-ca\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.140217 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.140146    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-sys\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.140340 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:35.140235    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 18 01:49:35.140340 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:35.140291    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-tls podName:1b769a28-9d0a-4189-9f5d-17eb826d0e8e nodeName:}" failed. No retries permitted until 2026-07-18 01:49:35.640272216 +0000 UTC m=+68.849986426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-tls") pod "node-exporter-rg7pk" (UID: "1b769a28-9d0a-4189-9f5d-17eb826d0e8e") : secret "node-exporter-tls" not found
Jul 18 01:49:35.140911 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.140597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.141490 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.141081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.141490 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:35.141412    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 18 01:49:35.141490 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:49:35.141467    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-tls podName:dc360c2e-bd30-45af-b4c9-9d1e280dfdf0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:35.641450642 +0000 UTC m=+68.851164864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-vhx57" (UID: "dc360c2e-bd30-45af-b4c9-9d1e280dfdf0") : secret "kube-state-metrics-tls" not found
Jul 18 01:49:35.147068 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.147021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.149133 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.149112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.149869 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.149822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84zn7\" (UniqueName: \"kubernetes.io/projected/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-api-access-84zn7\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.150657 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.150636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgwk2\" (UniqueName: \"kubernetes.io/projected/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-kube-api-access-lgwk2\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.272346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.272269    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.485216 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.485183    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.589077 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.589027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.645379 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.645348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.645525 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.645389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-tls\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.648864 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.648821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b769a28-9d0a-4189-9f5d-17eb826d0e8e-node-exporter-tls\") pod \"node-exporter-rg7pk\" (UID: \"1b769a28-9d0a-4189-9f5d-17eb826d0e8e\") " pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.649029 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.649004    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc360c2e-bd30-45af-b4c9-9d1e280dfdf0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vhx57\" (UID: \"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.695962 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.695902    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.841446 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.841367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-npj5j" event={"ID":"46d91c18-6dcf-4cb4-93a8-38d06290aa0a","Type":"ContainerStarted","Data":"76c4e291d01a631ee80ef37c7d7d944615a3b6a3c08b70bf4f1a489e50e9b636"}
Jul 18 01:49:35.841446 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.841416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-npj5j" event={"ID":"46d91c18-6dcf-4cb4-93a8-38d06290aa0a","Type":"ContainerStarted","Data":"c4e3dc6287040866dbc2a6023f101a0151d8467710584d4b07380d69e7ae3617"}
Jul 18 01:49:35.841607 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.841483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57"
Jul 18 01:49:35.865315 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.865254    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-npj5j" podStartSLOduration=66.745428242 podStartE2EDuration="1m7.865232688s" totalImagesPullingTime="1.119804446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:35.864387781 +0000 UTC m=+69.074102004" watchObservedRunningTime="2026-07-18 01:49:35.865232688 +0000 UTC m=+69.074946909"
Jul 18 01:49:35.874667 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.874641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rg7pk"
Jul 18 01:49:35.888894 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:35.888864    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b769a28_9d0a_4189_9f5d_17eb826d0e8e.slice/crio-b4055ad42fc1d4cdf043fb48f822be2d6d77b851713c06a22857d2f42aef1516 WatchSource:0}: Error finding container b4055ad42fc1d4cdf043fb48f822be2d6d77b851713c06a22857d2f42aef1516: Status 404 returned error can't find the container with id b4055ad42fc1d4cdf043fb48f822be2d6d77b851713c06a22857d2f42aef1516
Jul 18 01:49:35.899965 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.899929    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.991880 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.991803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.995979 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:35.995953    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:49:36.004813 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.004786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.007890 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.007817    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 18 01:49:36.007890 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.007850    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 18 01:49:36.009257 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.008169    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 18 01:49:36.009257 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.008463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 18 01:49:36.009257 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.008628    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 18 01:49:36.009257 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.008645    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 18 01:49:36.009257 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.008825    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 18 01:49:36.009257 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.008949    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 18 01:49:36.009518 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.009314    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 18 01:49:36.009650 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.009622    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x55z9\""
Jul 18 01:49:36.014094 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.014072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-vhx57"]
Jul 18 01:49:36.018420 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.018400    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:49:36.019815 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:36.019787    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc360c2e_bd30_45af_b4c9_9d1e280dfdf0.slice/crio-e646ab3231464b4ff8e869e1c4ea31e285ac90979556907fe631ba7cae218ec8 WatchSource:0}: Error finding container e646ab3231464b4ff8e869e1c4ea31e285ac90979556907fe631ba7cae218ec8: Status 404 returned error can't find the container with id e646ab3231464b4ff8e869e1c4ea31e285ac90979556907fe631ba7cae218ec8
Jul 18 01:49:36.118092 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.117990    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:36.150343 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqrqk\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-kube-api-access-dqrqk\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150526 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150404    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150526 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150526 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-config-out\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150526 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150506    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150728 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150728 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-web-config\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150728 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150728 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150728 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150728 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150994 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.150994 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.150803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.221263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.220589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:36.252122 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252312 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqrqk\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-kube-api-access-dqrqk\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252312 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252312 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252312 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-config-out\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252312 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252312 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252626 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-web-config\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252626 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252626 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252626 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252626 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.252626 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.252482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.254112 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.254085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.254569 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.254540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.254654 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.254600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.257186 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.257164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.258738 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.258718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.259681 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.259661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.259776 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.259743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.259776 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.259764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.259942 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.259917    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-config-out\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.260061 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.260028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-web-config\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.260492 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.260465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.261873 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.261849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.263981 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.263870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqrqk\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-kube-api-access-dqrqk\") pod \"alertmanager-main-0\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.319532 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.319499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:49:36.497911 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.497793    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:49:36.503525 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:36.503455    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0587e4cd_2989_4505_8042_a8953d4a817e.slice/crio-9a0daac24ac4e0087a1f2500d0d66bda2388af9dda0b8145ef552879be1cc21b WatchSource:0}: Error finding container 9a0daac24ac4e0087a1f2500d0d66bda2388af9dda0b8145ef552879be1cc21b: Status 404 returned error can't find the container with id 9a0daac24ac4e0087a1f2500d0d66bda2388af9dda0b8145ef552879be1cc21b
Jul 18 01:49:36.855862 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.855790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rg7pk" event={"ID":"1b769a28-9d0a-4189-9f5d-17eb826d0e8e","Type":"ContainerStarted","Data":"b4055ad42fc1d4cdf043fb48f822be2d6d77b851713c06a22857d2f42aef1516"}
Jul 18 01:49:36.858140 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.858105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerStarted","Data":"9a0daac24ac4e0087a1f2500d0d66bda2388af9dda0b8145ef552879be1cc21b"}
Jul 18 01:49:36.859533 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.859506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57" event={"ID":"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0","Type":"ContainerStarted","Data":"e646ab3231464b4ff8e869e1c4ea31e285ac90979556907fe631ba7cae218ec8"}
Jul 18 01:49:36.903246 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.901354    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-69dff55877-z9vlg"]
Jul 18 01:49:36.905258 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.905235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:36.908370 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.908343    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 18 01:49:36.908464 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.908410    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 18 01:49:36.908533 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.908456    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nhc5t\""
Jul 18 01:49:36.908703 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.908685    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 18 01:49:36.908762 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.908745    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c8iumvkgppmm6\""
Jul 18 01:49:36.908859 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.908845    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 18 01:49:36.908978 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.908955    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 18 01:49:36.918594 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.918574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69dff55877-z9vlg"]
Jul 18 01:49:36.958534 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.958507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:36.958705 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.958551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-metrics-client-ca\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:36.958705 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.958575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-tls\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:36.958705 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.958615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-grpc-tls\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:36.958705 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.958664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:36.959486 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.958730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whvfr\" (UniqueName: \"kubernetes.io/projected/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-kube-api-access-whvfr\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:36.959486 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.958766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:36.959486 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:36.958806    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.060505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.059874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.060505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.059937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-metrics-client-ca\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.060505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.059969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-tls\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.060505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.060013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-grpc-tls\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.060505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.060071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.060505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.060124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whvfr\" (UniqueName: \"kubernetes.io/projected/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-kube-api-access-whvfr\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.060505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.060157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.060505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.060182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.061811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.061782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-metrics-client-ca\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.064451 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.064425    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.065342 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.065317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-grpc-tls\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.066864 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.066805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-tls\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.067590 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.067568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.068441 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.068422    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.068755 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.068726    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.071785 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.071759    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whvfr\" (UniqueName: \"kubernetes.io/projected/f8ab39c5-194a-483c-98b2-2ab6a4f987f4-kube-api-access-whvfr\") pod \"thanos-querier-69dff55877-z9vlg\" (UID: \"f8ab39c5-194a-483c-98b2-2ab6a4f987f4\") " pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.219739 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.219704    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:37.802655 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.802622    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xrhq8"
Jul 18 01:49:37.888513 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.888476    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:37.992738 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:37.992699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:38.088418 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:38.088341    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:38.190644 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:38.190613    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:39.332642 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.332586    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6564b44b7-4btf9"]
Jul 18 01:49:39.335174 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.335153    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.338140 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.338114    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1kjdoffjcrdnn\""
Jul 18 01:49:39.339407 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.339290    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 18 01:49:39.339407 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.339390    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 18 01:49:39.339577 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.339410    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 18 01:49:39.339577 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.339380    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 18 01:49:39.339577 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.339391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xrrvf\""
Jul 18 01:49:39.347423 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.347400    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6564b44b7-4btf9"]
Jul 18 01:49:39.483710 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.483676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8543b4d1-c093-45d1-8df8-4a0be6775d93-secret-metrics-server-tls\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.483904 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.483730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8543b4d1-c093-45d1-8df8-4a0be6775d93-secret-metrics-server-client-certs\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.483904 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.483765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8543b4d1-c093-45d1-8df8-4a0be6775d93-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.483904 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.483846    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8543b4d1-c093-45d1-8df8-4a0be6775d93-client-ca-bundle\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.484101 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.483910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8543b4d1-c093-45d1-8df8-4a0be6775d93-audit-log\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.484101 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.483940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk598\" (UniqueName: \"kubernetes.io/projected/8543b4d1-c093-45d1-8df8-4a0be6775d93-kube-api-access-lk598\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.484101 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.483980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8543b4d1-c093-45d1-8df8-4a0be6775d93-metrics-server-audit-profiles\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.585093 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.584985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8543b4d1-c093-45d1-8df8-4a0be6775d93-secret-metrics-server-client-certs\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.585093 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.585056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8543b4d1-c093-45d1-8df8-4a0be6775d93-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.585309 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.585122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8543b4d1-c093-45d1-8df8-4a0be6775d93-client-ca-bundle\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.585309 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.585157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8543b4d1-c093-45d1-8df8-4a0be6775d93-audit-log\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.585309 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.585186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk598\" (UniqueName: \"kubernetes.io/projected/8543b4d1-c093-45d1-8df8-4a0be6775d93-kube-api-access-lk598\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.585309 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.585215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8543b4d1-c093-45d1-8df8-4a0be6775d93-metrics-server-audit-profiles\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.585309 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.585249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8543b4d1-c093-45d1-8df8-4a0be6775d93-secret-metrics-server-tls\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.585826 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.585791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8543b4d1-c093-45d1-8df8-4a0be6775d93-audit-log\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.585934 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.585801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8543b4d1-c093-45d1-8df8-4a0be6775d93-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.586789 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.586764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8543b4d1-c093-45d1-8df8-4a0be6775d93-metrics-server-audit-profiles\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.588177 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.588152    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8543b4d1-c093-45d1-8df8-4a0be6775d93-secret-metrics-server-client-certs\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.588267 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.588240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8543b4d1-c093-45d1-8df8-4a0be6775d93-secret-metrics-server-tls\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.588577 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.588557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8543b4d1-c093-45d1-8df8-4a0be6775d93-client-ca-bundle\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.593743 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.593721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk598\" (UniqueName: \"kubernetes.io/projected/8543b4d1-c093-45d1-8df8-4a0be6775d93-kube-api-access-lk598\") pod \"metrics-server-6564b44b7-4btf9\" (UID: \"8543b4d1-c093-45d1-8df8-4a0be6775d93\") " pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.647484 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.647455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:39.683988 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:39.683959    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:40.030090 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.030025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:40.764951 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.763774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6564b44b7-4btf9"]
Jul 18 01:49:40.771498 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:40.771456    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8543b4d1_c093_45d1_8df8_4a0be6775d93.slice/crio-b1064741ffcfa8fd624eaeb6409a04ea1abedc961f795ce0676c0fa95af694be WatchSource:0}: Error finding container b1064741ffcfa8fd624eaeb6409a04ea1abedc961f795ce0676c0fa95af694be: Status 404 returned error can't find the container with id b1064741ffcfa8fd624eaeb6409a04ea1abedc961f795ce0676c0fa95af694be
Jul 18 01:49:40.796707 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.796671    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69dff55877-z9vlg"]
Jul 18 01:49:40.877488 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.876646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lxbgv" event={"ID":"56718ed9-ac2a-4497-8484-20b905ba2022","Type":"ContainerStarted","Data":"94762dd06b335997036d00e6a03e7c6c75f21bd2569da4bd4bd2cdbb3379eced"}
Jul 18 01:49:40.877488 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.877434    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:49:40.883416 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.880619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rg7pk" event={"ID":"1b769a28-9d0a-4189-9f5d-17eb826d0e8e","Type":"ContainerStarted","Data":"4beb5678314775d52420484b2f70f06077e17c2a68202d4d8b7157bd11578948"}
Jul 18 01:49:40.884594 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.884566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg" event={"ID":"f8ab39c5-194a-483c-98b2-2ab6a4f987f4","Type":"ContainerStarted","Data":"9ea51cba4facb83f7f636c8d1e39ca8dba8ce59e536183d6af907565e018169f"}
Jul 18 01:49:40.890726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.890704    2580 generic.go:356] "Generic (PLEG): container finished" podID="0587e4cd-2989-4505-8042-a8953d4a817e" containerID="0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f" exitCode=0
Jul 18 01:49:40.891089 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.890791    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerDied","Data":"0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f"}
Jul 18 01:49:40.893517 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.893070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qj8km" event={"ID":"42e00b16-b4d4-4e92-8a85-b4477120c978","Type":"ContainerStarted","Data":"eda75c4684a0d7068abdb0e95c93116131234dc59e513317486c825d3a5e4462"}
Jul 18 01:49:40.897583 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.896978    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lxbgv" podStartSLOduration=66.824801448 podStartE2EDuration="1m12.896960339s" totalImagesPullingTime="6.072158891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:40.896262243 +0000 UTC m=+74.105976531" watchObservedRunningTime="2026-07-18 01:49:40.896960339 +0000 UTC m=+74.106674560"
Jul 18 01:49:40.899132 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.899022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57" event={"ID":"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0","Type":"ContainerStarted","Data":"71e2b18e50e0fde49478e2a7962c8f8d5f4da72d530772a4036af34151e21523"}
Jul 18 01:49:40.899132 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.899086    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57" event={"ID":"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0","Type":"ContainerStarted","Data":"1f8f8630f5e58738e3525b7ac567a76767ade015956fa97793deec8cd6f31493"}
Jul 18 01:49:40.900759 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.900668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6564b44b7-4btf9" event={"ID":"8543b4d1-c093-45d1-8df8-4a0be6775d93","Type":"ContainerStarted","Data":"b1064741ffcfa8fd624eaeb6409a04ea1abedc961f795ce0676c0fa95af694be"}
Jul 18 01:49:40.913752 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:40.913713    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qj8km" podStartSLOduration=66.801717281 podStartE2EDuration="1m12.913699142s" totalImagesPullingTime="6.111981861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:40.913496943 +0000 UTC m=+74.123211166" watchObservedRunningTime="2026-07-18 01:49:40.913699142 +0000 UTC m=+74.123413340"
Jul 18 01:49:41.189006 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.188967    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:41.192827 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.192791    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:49:41.198608 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.198577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.203713 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.203693    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 18 01:49:41.207085 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.204715    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 18 01:49:41.207085 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.204964    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 18 01:49:41.207085 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.205205    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 18 01:49:41.207085 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.205383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 18 01:49:41.207085 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.205612    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 18 01:49:41.207085 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.205811    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 18 01:49:41.207085 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.206010    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 18 01:49:41.207085 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.206290    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 18 01:49:41.207570 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.207281    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 18 01:49:41.207570 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.207312    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 18 01:49:41.207570 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.207534    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 18 01:49:41.207852 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.207832    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xdgtp\""
Jul 18 01:49:41.217033 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.208084    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5klgan7bk4mm8\""
Jul 18 01:49:41.217033 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.208229    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 18 01:49:41.217033 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.213116    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 18 01:49:41.217033 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.216872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:49:41.271924 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.271841    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btppm\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-kube-api-access-btppm\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-config\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.300988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.301415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.301017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.303067 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.301077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.303067 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.301111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.303067 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.301144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.303067 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.301194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.372674 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.372639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.401991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btppm\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-kube-api-access-btppm\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-config\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.403263 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.402504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.405130 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.404839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.405130 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.404888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.405130 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.404922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.413204 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.410713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.416938 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.413755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.416938 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.414808    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.416938 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.415397    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.416938 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.415880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.416938 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.416306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.417456 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.417430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.420069 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.419386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.421719 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.421686    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.422117 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.422093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.422571 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.422546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.422798 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.422782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.423259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.423237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.427621 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.427601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btppm\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-kube-api-access-btppm\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.429425 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.429396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.430849 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.430788    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.433074 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.432522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.433074 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.432879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-config\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.433440 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.433378    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.434847 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.434828    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.523428 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.523197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:41.717469 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.717332    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:49:41.722359 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:49:41.722325    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f708533_85c5_4b80_9fd6_fe01d167da3d.slice/crio-84037ddfebb4570c597e655489498d91dccb9bf9e6b313c1f1a642040830b222 WatchSource:0}: Error finding container 84037ddfebb4570c597e655489498d91dccb9bf9e6b313c1f1a642040830b222: Status 404 returned error can't find the container with id 84037ddfebb4570c597e655489498d91dccb9bf9e6b313c1f1a642040830b222
Jul 18 01:49:41.909240 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.909204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57" event={"ID":"dc360c2e-bd30-45af-b4c9-9d1e280dfdf0","Type":"ContainerStarted","Data":"1581e9925b1edcf20f1c51423f6bb6784fdb70027b12f15d6124d7643ae3b5f1"}
Jul 18 01:49:41.911959 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.911931    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerID="e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a" exitCode=0
Jul 18 01:49:41.912122 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.911986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerDied","Data":"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"}
Jul 18 01:49:41.912122 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.912083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerStarted","Data":"84037ddfebb4570c597e655489498d91dccb9bf9e6b313c1f1a642040830b222"}
Jul 18 01:49:41.915461 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.915435    2580 generic.go:356] "Generic (PLEG): container finished" podID="1b769a28-9d0a-4189-9f5d-17eb826d0e8e" containerID="4beb5678314775d52420484b2f70f06077e17c2a68202d4d8b7157bd11578948" exitCode=0
Jul 18 01:49:41.915574 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.915547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rg7pk" event={"ID":"1b769a28-9d0a-4189-9f5d-17eb826d0e8e","Type":"ContainerDied","Data":"4beb5678314775d52420484b2f70f06077e17c2a68202d4d8b7157bd11578948"}
Jul 18 01:49:41.931567 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:41.930281    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-vhx57" podStartSLOduration=3.369945956 podStartE2EDuration="7.930261407s" totalImagesPullingTime="4.560315451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:41.92884183 +0000 UTC m=+75.138556063" watchObservedRunningTime="2026-07-18 01:49:41.930261407 +0000 UTC m=+75.139975627"
Jul 18 01:49:42.927288 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:42.926453    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rg7pk" event={"ID":"1b769a28-9d0a-4189-9f5d-17eb826d0e8e","Type":"ContainerStarted","Data":"a8572e3386317f2bea6aca2eea53fb475372c29a12e4bd0e0828d3aa5c1b6e1d"}
Jul 18 01:49:43.935062 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.935007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6564b44b7-4btf9" event={"ID":"8543b4d1-c093-45d1-8df8-4a0be6775d93","Type":"ContainerStarted","Data":"8aafc0ef826ed8583c4622ee7fa9c5ee89a1f21ed3c1d2a9f394c5176816b5ab"}
Jul 18 01:49:43.937699 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.937669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rg7pk" event={"ID":"1b769a28-9d0a-4189-9f5d-17eb826d0e8e","Type":"ContainerStarted","Data":"c7bd60de38c73c4a69f45b86063fe5ec3b73c8a3346d5ed757ab9106289ea7bf"}
Jul 18 01:49:43.940489 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.940400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg" event={"ID":"f8ab39c5-194a-483c-98b2-2ab6a4f987f4","Type":"ContainerStarted","Data":"f171b81277ce5c85108551c99af1f48740ba4310057238cb10564234f967a671"}
Jul 18 01:49:43.940489 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.940450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg" event={"ID":"f8ab39c5-194a-483c-98b2-2ab6a4f987f4","Type":"ContainerStarted","Data":"d32d05a4c2cfcaeb94502c7ad888e0c399a73d0962983718ede9dd27b8c6c7f1"}
Jul 18 01:49:43.940613 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.940489    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg" event={"ID":"f8ab39c5-194a-483c-98b2-2ab6a4f987f4","Type":"ContainerStarted","Data":"5c8ebdb993ef046cd2fa0c2e35f737fa9e15409cce10b72236e159571c6c4f26"}
Jul 18 01:49:43.943703 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.943683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerStarted","Data":"9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580"}
Jul 18 01:49:43.943780 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.943745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerStarted","Data":"171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412"}
Jul 18 01:49:43.943829 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.943783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerStarted","Data":"38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8"}
Jul 18 01:49:43.943829 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.943819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerStarted","Data":"a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b"}
Jul 18 01:49:43.955718 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.955670    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6564b44b7-4btf9" podStartSLOduration=2.237346368 podStartE2EDuration="4.955653413s" totalImagesPullingTime="2.718307045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:43.955336475 +0000 UTC m=+77.165050713" watchObservedRunningTime="2026-07-18 01:49:43.955653413 +0000 UTC m=+77.165367636"
Jul 18 01:49:43.983440 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:43.983271    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rg7pk" podStartSLOduration=5.165350816 podStartE2EDuration="9.983255183s" totalImagesPullingTime="4.690352344s" totalInitContainerRuntime="127.552023ms" isStatefulPod=true podCreationTimestamp="2026-07-18 01:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:43.9818444 +0000 UTC m=+77.191558624" watchObservedRunningTime="2026-07-18 01:49:43.983255183 +0000 UTC m=+77.192969403"
Jul 18 01:49:44.491245 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:44.491209    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:44.591438 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:44.591403    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:44.703032 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:44.702997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:44.804405 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:44.804311    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:44.864626 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:44.864588    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:44.953692 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:44.953657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerStarted","Data":"2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47"}
Jul 18 01:49:44.988480 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:44.988448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:45.375607 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.375569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:45.476209 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.476178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:45.776348 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.776270    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:45.884708 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.884684    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:45.962670 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.962637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerStarted","Data":"6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09"}
Jul 18 01:49:45.966750 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.966715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerStarted","Data":"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"}
Jul 18 01:49:45.966874 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.966799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerStarted","Data":"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"}
Jul 18 01:49:45.966874 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.966842    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerStarted","Data":"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"}
Jul 18 01:49:45.966991 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.966877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerStarted","Data":"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"}
Jul 18 01:49:45.966991 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.966925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerStarted","Data":"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"}
Jul 18 01:49:45.970063 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.970021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg" event={"ID":"f8ab39c5-194a-483c-98b2-2ab6a4f987f4","Type":"ContainerStarted","Data":"32d22992f308b5f41301e928f965a6e4b386954b917afe742cb5431e7677affd"}
Jul 18 01:49:45.970156 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.970100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg" event={"ID":"f8ab39c5-194a-483c-98b2-2ab6a4f987f4","Type":"ContainerStarted","Data":"d2eda2ecdf1a937e9e7096c698f347a38ecc5ad75383508638876bef743592d3"}
Jul 18 01:49:45.970156 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.970142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg" event={"ID":"f8ab39c5-194a-483c-98b2-2ab6a4f987f4","Type":"ContainerStarted","Data":"dd5fa020f6611652e18b5964c67713d9b6f2c18a5846112159dd107580e01f60"}
Jul 18 01:49:45.970244 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.970209    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:45.993857 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:45.993764    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.742625108 podStartE2EDuration="10.993745376s" totalImagesPullingTime="8.224668985s" totalInitContainerRuntime="26.451283ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:45.991376696 +0000 UTC m=+79.201090945" watchObservedRunningTime="2026-07-18 01:49:45.993745376 +0000 UTC m=+79.203459597"
Jul 18 01:49:46.016806 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:46.016751    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg" podStartSLOduration=5.658059127 podStartE2EDuration="10.016735008s" totalImagesPullingTime="4.358675881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:46.014222953 +0000 UTC m=+79.223937179" watchObservedRunningTime="2026-07-18 01:49:46.016735008 +0000 UTC m=+79.226449280"
Jul 18 01:49:46.316859 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:46.316573    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.416934 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:46.416895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.483817 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:46.483785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.584441 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:46.584363    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.709083 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:46.709038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.810773 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:46.810729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.908860 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:46.908780    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.977778 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:46.977737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerStarted","Data":"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"}
Jul 18 01:49:47.009575 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:47.009525    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.424115921 podStartE2EDuration="6.009507527s" totalImagesPullingTime="3.558466906s" totalInitContainerRuntime="26.9247ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:47.007026203 +0000 UTC m=+80.216740423" watchObservedRunningTime="2026-07-18 01:49:47.009507527 +0000 UTC m=+80.219221771"
Jul 18 01:49:47.603813 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:47.603698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:47.665698 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:47.665664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:49.335735 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:49.335697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:49.435598 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:49.435568    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:51.032888 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:51.032847    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:51.134115 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:51.134082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:51.194550 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:51.194522    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:51.524206 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:51.524174    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:49:51.984268 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:51.984245    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-69dff55877-z9vlg"
Jul 18 01:49:55.274229 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:55.274191    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:57.019486 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:57.019444    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:57.103795 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:57.103756    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:58.412578 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:58.412540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:58.513305 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:58.513266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:59.648112 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:59.648076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:49:59.648500 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:49:59.648139    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:50:00.408504 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:00.408476    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:00.467163 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:00.467128    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:05.083900 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:05.083871    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:05.141551 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:05.141520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:06.017411 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:06.017382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:06.071729 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:06.071697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:07.410841 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:07.410807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:07.511401 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:07.511373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:12.929535 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:12.929500    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lxbgv"
Jul 18 01:50:13.602546 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:13.602513    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:13.657916 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:13.657885    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:15.399424 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:15.399388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:19.653550 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:19.653519    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:50:19.657309 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:19.657285    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6564b44b7-4btf9"
Jul 18 01:50:20.356297 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:20.356267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:21.011826 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:21.011794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:21.112063 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:21.112009    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:21.605429 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:21.605400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:36.016259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:36.016225    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:36.073919 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:36.073889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:41.524191 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:41.524155    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:50:41.544283 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:41.544260    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:50:42.207203 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:42.207176    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:50:54.195488 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:54.195445    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:54.756955 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:54.756894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:55.244280 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:55.244246    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:50:55.245828 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:55.245562    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="alertmanager" containerID="cri-o://a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b" gracePeriod=120
Jul 18 01:50:55.245828 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:55.245590    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy-web" containerID="cri-o://171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412" gracePeriod=120
Jul 18 01:50:55.245828 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:55.245590    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy-metric" containerID="cri-o://2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47" gracePeriod=120
Jul 18 01:50:55.245828 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:55.245631    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="prom-label-proxy" containerID="cri-o://6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09" gracePeriod=120
Jul 18 01:50:55.245828 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:55.245711    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy" containerID="cri-o://9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580" gracePeriod=120
Jul 18 01:50:55.245828 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:55.245612    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="config-reloader" containerID="cri-o://38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8" gracePeriod=120
Jul 18 01:50:56.255566 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255532    2580 generic.go:356] "Generic (PLEG): container finished" podID="0587e4cd-2989-4505-8042-a8953d4a817e" containerID="6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09" exitCode=0
Jul 18 01:50:56.255566 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255555    2580 generic.go:356] "Generic (PLEG): container finished" podID="0587e4cd-2989-4505-8042-a8953d4a817e" containerID="2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47" exitCode=0
Jul 18 01:50:56.255566 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255562    2580 generic.go:356] "Generic (PLEG): container finished" podID="0587e4cd-2989-4505-8042-a8953d4a817e" containerID="9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580" exitCode=0
Jul 18 01:50:56.255566 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255567    2580 generic.go:356] "Generic (PLEG): container finished" podID="0587e4cd-2989-4505-8042-a8953d4a817e" containerID="38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8" exitCode=0
Jul 18 01:50:56.255566 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255573    2580 generic.go:356] "Generic (PLEG): container finished" podID="0587e4cd-2989-4505-8042-a8953d4a817e" containerID="a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b" exitCode=0
Jul 18 01:50:56.256077 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerDied","Data":"6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09"}
Jul 18 01:50:56.256077 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerDied","Data":"2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47"}
Jul 18 01:50:56.256077 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerDied","Data":"9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580"}
Jul 18 01:50:56.256077 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerDied","Data":"38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8"}
Jul 18 01:50:56.256077 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.255747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerDied","Data":"a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b"}
Jul 18 01:50:56.485775 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.485751    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:56.538255 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538175    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqrqk\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-kube-api-access-dqrqk\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538255 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538235    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-trusted-ca-bundle\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538465 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538399    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-cluster-tls-config\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538465 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-config-volume\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538569 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538484    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-main-tls\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538569 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538520    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-web\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538569 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538553    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-tls-assets\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538719 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538584    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-web-config\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538719 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538615    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-config-out\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538719 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538616    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:56.538719 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538647    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538719 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538682    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-main-db\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538945 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-metrics-client-ca\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.538945 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.538796    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy\") pod \"0587e4cd-2989-4505-8042-a8953d4a817e\" (UID: \"0587e4cd-2989-4505-8042-a8953d4a817e\") "
Jul 18 01:50:56.539395 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.539089    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.539678 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.539651    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:50:56.540087 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.540058    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:56.542942 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.542914    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-config-volume" (OuterVolumeSpecName: "config-volume") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:56.543074 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.542958    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-kube-api-access-dqrqk" (OuterVolumeSpecName: "kube-api-access-dqrqk") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "kube-api-access-dqrqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:50:56.543183 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.543159    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:50:56.543419 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.543380    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:56.544363 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.544337    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:56.544582 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.544545    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:56.544650 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.544611    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:56.544763 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.544745    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-config-out" (OuterVolumeSpecName: "config-out") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:50:56.548605 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.548582    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:56.555472 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.555448    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-web-config" (OuterVolumeSpecName: "web-config") pod "0587e4cd-2989-4505-8042-a8953d4a817e" (UID: "0587e4cd-2989-4505-8042-a8953d4a817e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:56.639422 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639391    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0587e4cd-2989-4505-8042-a8953d4a817e-metrics-client-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639422 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639422    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639435    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqrqk\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-kube-api-access-dqrqk\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639448    2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-cluster-tls-config\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639459    2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-config-volume\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639470    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-main-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639482    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639494    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0587e4cd-2989-4505-8042-a8953d4a817e-tls-assets\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639504    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-web-config\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639514    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-config-out\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639525    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0587e4cd-2989-4505-8042-a8953d4a817e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:56.639615 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:56.639539    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0587e4cd-2989-4505-8042-a8953d4a817e-alertmanager-main-db\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:57.261946 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.261912    2580 generic.go:356] "Generic (PLEG): container finished" podID="0587e4cd-2989-4505-8042-a8953d4a817e" containerID="171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412" exitCode=0
Jul 18 01:50:57.262443 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.261996    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.262443 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.262001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerDied","Data":"171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412"}
Jul 18 01:50:57.262443 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.262118    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0587e4cd-2989-4505-8042-a8953d4a817e","Type":"ContainerDied","Data":"9a0daac24ac4e0087a1f2500d0d66bda2388af9dda0b8145ef552879be1cc21b"}
Jul 18 01:50:57.262443 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.262187    2580 scope.go:117] "RemoveContainer" containerID="6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09"
Jul 18 01:50:57.270886 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.270871    2580 scope.go:117] "RemoveContainer" containerID="2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47"
Jul 18 01:50:57.278446 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.278426    2580 scope.go:117] "RemoveContainer" containerID="9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580"
Jul 18 01:50:57.286417 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.286368    2580 scope.go:117] "RemoveContainer" containerID="171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412"
Jul 18 01:50:57.289036 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.289017    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:50:57.293685 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.293662    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:50:57.295857 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.295839    2580 scope.go:117] "RemoveContainer" containerID="38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8"
Jul 18 01:50:57.303666 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.303646    2580 scope.go:117] "RemoveContainer" containerID="a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b"
Jul 18 01:50:57.310399 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.310380    2580 scope.go:117] "RemoveContainer" containerID="0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f"
Jul 18 01:50:57.318178 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.318153    2580 scope.go:117] "RemoveContainer" containerID="6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09"
Jul 18 01:50:57.318443 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:50:57.318423    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09\": container with ID starting with 6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09 not found: ID does not exist" containerID="6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09"
Jul 18 01:50:57.318513 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.318451    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09"} err="failed to get container status \"6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09\": rpc error: code = NotFound desc = could not find container \"6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09\": container with ID starting with 6a772d9bf6b3ed244993d4ab6c07aa6a046cb3d03955e6a1c08e26a2877ccf09 not found: ID does not exist"
Jul 18 01:50:57.318513 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.318487    2580 scope.go:117] "RemoveContainer" containerID="2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47"
Jul 18 01:50:57.318732 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:50:57.318717    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47\": container with ID starting with 2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47 not found: ID does not exist" containerID="2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47"
Jul 18 01:50:57.318776 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.318735    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47"} err="failed to get container status \"2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47\": rpc error: code = NotFound desc = could not find container \"2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47\": container with ID starting with 2da067cf189719ce2552ee1603c5c32666f8514e57ef87c9c1d52fe8e477cd47 not found: ID does not exist"
Jul 18 01:50:57.318776 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.318747    2580 scope.go:117] "RemoveContainer" containerID="9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580"
Jul 18 01:50:57.319087 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:50:57.319017    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580\": container with ID starting with 9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580 not found: ID does not exist" containerID="9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580"
Jul 18 01:50:57.319207 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.319090    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580"} err="failed to get container status \"9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580\": rpc error: code = NotFound desc = could not find container \"9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580\": container with ID starting with 9a2ac18d3a730152924ec6ffb82f1de443eecc21562dfc3ac64a115010090580 not found: ID does not exist"
Jul 18 01:50:57.319207 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.319114    2580 scope.go:117] "RemoveContainer" containerID="171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412"
Jul 18 01:50:57.319797 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:50:57.319659    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412\": container with ID starting with 171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412 not found: ID does not exist" containerID="171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412"
Jul 18 01:50:57.319797 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.319693    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412"} err="failed to get container status \"171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412\": rpc error: code = NotFound desc = could not find container \"171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412\": container with ID starting with 171b3090e67553cd144f8d3ed1710f8d5b31106e5f4f9090e9e12f298363c412 not found: ID does not exist"
Jul 18 01:50:57.319797 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.319717    2580 scope.go:117] "RemoveContainer" containerID="38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8"
Jul 18 01:50:57.320392 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:50:57.320357    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8\": container with ID starting with 38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8 not found: ID does not exist" containerID="38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8"
Jul 18 01:50:57.320559 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.320521    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8"} err="failed to get container status \"38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8\": rpc error: code = NotFound desc = could not find container \"38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8\": container with ID starting with 38f5932d82ed6495e9dede57a72d41f63ff27d9d64b044f0f9eba99e24229aa8 not found: ID does not exist"
Jul 18 01:50:57.320645 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.320624    2580 scope.go:117] "RemoveContainer" containerID="a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b"
Jul 18 01:50:57.321013 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:50:57.320987    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b\": container with ID starting with a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b not found: ID does not exist" containerID="a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b"
Jul 18 01:50:57.321104 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321016    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b"} err="failed to get container status \"a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b\": rpc error: code = NotFound desc = could not find container \"a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b\": container with ID starting with a59e49aefa26c21374a371a94c85ec56ff0ae501b06c643a32c55aaa14096b2b not found: ID does not exist"
Jul 18 01:50:57.321104 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321031    2580 scope.go:117] "RemoveContainer" containerID="0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f"
Jul 18 01:50:57.321279 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:50:57.321263    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f\": container with ID starting with 0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f not found: ID does not exist" containerID="0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f"
Jul 18 01:50:57.321319 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321282    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f"} err="failed to get container status \"0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f\": rpc error: code = NotFound desc = could not find container \"0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f\": container with ID starting with 0a6a1dfa3ff903d8288db635eafb7d466e8400a9bdf72857a5fd5f9121316c1f not found: ID does not exist"
Jul 18 01:50:57.321600 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321582    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:50:57.321903 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321887    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy"
Jul 18 01:50:57.321903 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321903    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321913    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="config-reloader"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321919    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="config-reloader"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321925    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy-metric"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321930    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy-metric"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321939    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="prom-label-proxy"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321944    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="prom-label-proxy"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321954    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy-web"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321963    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy-web"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321974    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="init-config-reloader"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321979    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="init-config-reloader"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321988    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="alertmanager"
Jul 18 01:50:57.322007 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.321996    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="alertmanager"
Jul 18 01:50:57.322426 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.322036    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy-web"
Jul 18 01:50:57.322426 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.322132    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="alertmanager"
Jul 18 01:50:57.322426 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.322141    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="config-reloader"
Jul 18 01:50:57.322426 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.322148    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="prom-label-proxy"
Jul 18 01:50:57.322426 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.322154    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy"
Jul 18 01:50:57.322426 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.322160    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" containerName="kube-rbac-proxy-metric"
Jul 18 01:50:57.326687 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.326670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.329537 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.329518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 18 01:50:57.329618 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.329520    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 18 01:50:57.329820 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.329791    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 18 01:50:57.329820 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.329803    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 18 01:50:57.329820 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.329791    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 18 01:50:57.330142 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.330124    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 18 01:50:57.330142 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.330133    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 18 01:50:57.330429 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.330415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x55z9\""
Jul 18 01:50:57.331149 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.331128    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 18 01:50:57.340358 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.340333    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 18 01:50:57.341328 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.341305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:50:57.345384 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/991e70a6-ba90-465e-88a4-2fa8c000fe77-config-out\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345471 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345471 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345559 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdw8q\" (UniqueName: \"kubernetes.io/projected/991e70a6-ba90-465e-88a4-2fa8c000fe77-kube-api-access-qdw8q\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345559 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/991e70a6-ba90-465e-88a4-2fa8c000fe77-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345625 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345660 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345660 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-web-config\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345725 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/991e70a6-ba90-465e-88a4-2fa8c000fe77-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345763 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/991e70a6-ba90-465e-88a4-2fa8c000fe77-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345795 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345773    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345827 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/991e70a6-ba90-465e-88a4-2fa8c000fe77-tls-assets\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.345827 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.345813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-config-volume\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.390024 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.389991    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0587e4cd-2989-4505-8042-a8953d4a817e" path="/var/lib/kubelet/pods/0587e4cd-2989-4505-8042-a8953d4a817e/volumes"
Jul 18 01:50:57.446478 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/991e70a6-ba90-465e-88a4-2fa8c000fe77-config-out\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446629 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446629 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446629 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdw8q\" (UniqueName: \"kubernetes.io/projected/991e70a6-ba90-465e-88a4-2fa8c000fe77-kube-api-access-qdw8q\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446629 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/991e70a6-ba90-465e-88a4-2fa8c000fe77-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446629 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446872 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446872 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-web-config\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446872 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/991e70a6-ba90-465e-88a4-2fa8c000fe77-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446872 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/991e70a6-ba90-465e-88a4-2fa8c000fe77-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446872 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446872 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/991e70a6-ba90-465e-88a4-2fa8c000fe77-tls-assets\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.446872 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.446799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-config-volume\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.447580 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.447442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/991e70a6-ba90-465e-88a4-2fa8c000fe77-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.447735 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.447708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/991e70a6-ba90-465e-88a4-2fa8c000fe77-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.447803 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.447788    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/991e70a6-ba90-465e-88a4-2fa8c000fe77-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.450440 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.449999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-config-volume\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.450440 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.450169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.450440 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.450391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/991e70a6-ba90-465e-88a4-2fa8c000fe77-tls-assets\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.450440 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.450410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-web-config\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.450724 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.450702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.450769 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.450728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.451069 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.451013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.451183 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.451166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/991e70a6-ba90-465e-88a4-2fa8c000fe77-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.451438 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.451418    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/991e70a6-ba90-465e-88a4-2fa8c000fe77-config-out\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.454559 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.454537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdw8q\" (UniqueName: \"kubernetes.io/projected/991e70a6-ba90-465e-88a4-2fa8c000fe77-kube-api-access-qdw8q\") pod \"alertmanager-main-0\" (UID: \"991e70a6-ba90-465e-88a4-2fa8c000fe77\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.642308 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.642216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:50:57.776802 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:57.776638    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:50:57.779355 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:50:57.779326    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991e70a6_ba90_465e_88a4_2fa8c000fe77.slice/crio-66de5aa76aa50e11933735dfd178fabc5c64faeab24871fb64637ed60954230a WatchSource:0}: Error finding container 66de5aa76aa50e11933735dfd178fabc5c64faeab24871fb64637ed60954230a: Status 404 returned error can't find the container with id 66de5aa76aa50e11933735dfd178fabc5c64faeab24871fb64637ed60954230a
Jul 18 01:50:58.267517 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:58.267481    2580 generic.go:356] "Generic (PLEG): container finished" podID="991e70a6-ba90-465e-88a4-2fa8c000fe77" containerID="4b1ee5f81262c55f8fe8db0d42cab4c74d02545ea27a0bd440ad37500a856ea5" exitCode=0
Jul 18 01:50:58.267897 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:58.267566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"991e70a6-ba90-465e-88a4-2fa8c000fe77","Type":"ContainerDied","Data":"4b1ee5f81262c55f8fe8db0d42cab4c74d02545ea27a0bd440ad37500a856ea5"}
Jul 18 01:50:58.267897 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:58.267635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"991e70a6-ba90-465e-88a4-2fa8c000fe77","Type":"ContainerStarted","Data":"66de5aa76aa50e11933735dfd178fabc5c64faeab24871fb64637ed60954230a"}
Jul 18 01:50:59.277130 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.277096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"991e70a6-ba90-465e-88a4-2fa8c000fe77","Type":"ContainerStarted","Data":"3c8b7c8077569539ca4ed27518a78f468af98d75dd6ec7b485c26627efa9b851"}
Jul 18 01:50:59.277654 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.277164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"991e70a6-ba90-465e-88a4-2fa8c000fe77","Type":"ContainerStarted","Data":"ccd6336e675e5725ad8b5e000f299f0ae507c0ce4396b57b977bcf2a3617f293"}
Jul 18 01:50:59.277654 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.277209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"991e70a6-ba90-465e-88a4-2fa8c000fe77","Type":"ContainerStarted","Data":"8c212b2e654b473f9cd2d7a0dc7b2bbf599724983629bc0af15f0b294c2f54d1"}
Jul 18 01:50:59.277654 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.277247    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"991e70a6-ba90-465e-88a4-2fa8c000fe77","Type":"ContainerStarted","Data":"882283c38d64bba421c76c2e750916731ca0c8f3a32819d691000d0c18e3cda8"}
Jul 18 01:50:59.277654 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.277282    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"991e70a6-ba90-465e-88a4-2fa8c000fe77","Type":"ContainerStarted","Data":"49f260f749a693b5ed2f08f84c0a9cf2f9425de2ef7d21518da08a578bd67205"}
Jul 18 01:50:59.277654 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.277321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"991e70a6-ba90-465e-88a4-2fa8c000fe77","Type":"ContainerStarted","Data":"b2f75b795eb79a8b723c8d5d8303dd10e6578aa728f8f43a0100d7accaac15bf"}
Jul 18 01:50:59.312753 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.312700    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.297405211 podStartE2EDuration="2.312671769s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.266558ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:50:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:59.311777317 +0000 UTC m=+152.521491540" watchObservedRunningTime="2026-07-18 01:50:59.312671769 +0000 UTC m=+152.522385989"
Jul 18 01:50:59.571754 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.571644    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:50:59.572955 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.572344    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy" containerID="cri-o://cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903" gracePeriod=600
Jul 18 01:50:59.572955 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.572554    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c" gracePeriod=600
Jul 18 01:50:59.572955 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.572929    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="config-reloader" containerID="cri-o://3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d" gracePeriod=600
Jul 18 01:50:59.573714 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.573688    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="prometheus" containerID="cri-o://a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302" gracePeriod=600
Jul 18 01:50:59.573915 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.573596    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy-web" containerID="cri-o://f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6" gracePeriod=600
Jul 18 01:50:59.574211 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.573487    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="thanos-sidecar" containerID="cri-o://148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787" gracePeriod=600
Jul 18 01:50:59.839469 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.839444    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:50:59.864392 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864356    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-config-out\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864542 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864521    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-trusted-ca-bundle\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864604 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864557    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864604 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864591    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-thanos-prometheus-http-client-file\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864723 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-metrics-client-certs\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864723 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864723 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864673    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-web-config\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864723 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864711    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-tls-assets\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864747    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-grpc-tls\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864772    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btppm\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-kube-api-access-btppm\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864815    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-config\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864858    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-0\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.864912 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864887    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-kube-rbac-proxy\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.865172 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-1\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.865172 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864914    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:59.865172 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864943    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-db\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.865172 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.864979    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-kubelet-serving-ca-bundle\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.865172 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.865006    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-metrics-client-ca\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.865172 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.865035    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-2\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.865172 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.865084    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-tls\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.865474 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.865346    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.867370 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.865528    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:59.867370 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.866980    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:50:59.868770 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.868264    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:59.868770 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.868570    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:59.869123 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.869076    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:59.870252 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.870137    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:59.870252 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.870214    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-kube-api-access-btppm" (OuterVolumeSpecName: "kube-api-access-btppm") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "kube-api-access-btppm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:50:59.871721 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.870953    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:59.872310 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.872274    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:50:59.873463 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.873425    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-config" (OuterVolumeSpecName: "config") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:59.873651 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.873596    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:59.873651 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.873612    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:59.873974 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.873929    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:59.874736 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.874696    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-config-out" (OuterVolumeSpecName: "config-out") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:50:59.875334 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.875294    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:59.876911 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.876875    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:59.877009 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.876907    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:59.888529 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.888499    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-web-config" (OuterVolumeSpecName: "web-config") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:59.965849 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.965812    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-serving-certs-ca-bundle\") pod \"2f708533-85c5-4b80-9fd6-fe01d167da3d\" (UID: \"2f708533-85c5-4b80-9fd6-fe01d167da3d\") "
Jul 18 01:50:59.966005 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.965959    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966005 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.965976    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966005 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.965988    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-metrics-client-certs\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966005 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.965997    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966007    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-web-config\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966016    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-tls-assets\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966024    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-grpc-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966031    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btppm\" (UniqueName: \"kubernetes.io/projected/2f708533-85c5-4b80-9fd6-fe01d167da3d-kube-api-access-btppm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966064    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-config\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966074    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966083    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-kube-rbac-proxy\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966091    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966100    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-db\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966109    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966120    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-metrics-client-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966135    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966129    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2f708533-85c5-4b80-9fd6-fe01d167da3d" (UID: "2f708533-85c5-4b80-9fd6-fe01d167da3d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966144    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f708533-85c5-4b80-9fd6-fe01d167da3d-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:50:59.966166 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:50:59.966153    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f708533-85c5-4b80-9fd6-fe01d167da3d-config-out\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:51:00.067241 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.067201    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f708533-85c5-4b80-9fd6-fe01d167da3d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:51:00.284655 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284616    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerID="d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c" exitCode=0
Jul 18 01:51:00.284655 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284649    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerID="cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903" exitCode=0
Jul 18 01:51:00.284655 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284659    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerID="f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6" exitCode=0
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284671    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerID="148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787" exitCode=0
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284678    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerID="3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d" exitCode=0
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284687    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerID="a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302" exitCode=0
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerDied","Data":"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"}
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284712    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284812    2580 scope.go:117] "RemoveContainer" containerID="d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerDied","Data":"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"}
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerDied","Data":"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"}
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerDied","Data":"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"}
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.284999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerDied","Data":"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"}
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.285037    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerDied","Data":"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"}
Jul 18 01:51:00.285119 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.285089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f708533-85c5-4b80-9fd6-fe01d167da3d","Type":"ContainerDied","Data":"84037ddfebb4570c597e655489498d91dccb9bf9e6b313c1f1a642040830b222"}
Jul 18 01:51:00.293238 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.293218    2580 scope.go:117] "RemoveContainer" containerID="cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"
Jul 18 01:51:00.300123 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.300105    2580 scope.go:117] "RemoveContainer" containerID="f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"
Jul 18 01:51:00.307098 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.307070    2580 scope.go:117] "RemoveContainer" containerID="148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"
Jul 18 01:51:00.308548 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.308527    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:51:00.312924 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.312905    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:51:00.326309 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.326294    2580 scope.go:117] "RemoveContainer" containerID="3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"
Jul 18 01:51:00.333537 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.333521    2580 scope.go:117] "RemoveContainer" containerID="a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"
Jul 18 01:51:00.341367 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341346    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:51:00.341641 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341622    2580 scope.go:117] "RemoveContainer" containerID="e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"
Jul 18 01:51:00.341746 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341727    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="thanos-sidecar"
Jul 18 01:51:00.341811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341747    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="thanos-sidecar"
Jul 18 01:51:00.341811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341760    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy-thanos"
Jul 18 01:51:00.341811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341769    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy-thanos"
Jul 18 01:51:00.341811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341792    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="config-reloader"
Jul 18 01:51:00.341811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341801    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="config-reloader"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341813    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy-web"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341821    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy-web"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341845    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="init-config-reloader"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341854    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="init-config-reloader"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341863    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341871    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341882    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="prometheus"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341890    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="prometheus"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341957    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="thanos-sidecar"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341970    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="prometheus"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341982    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="config-reloader"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.341991    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy-web"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.342001    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy"
Jul 18 01:51:00.342084 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.342009    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" containerName="kube-rbac-proxy-thanos"
Jul 18 01:51:00.345800 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.345781    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.348988 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.348945    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 18 01:51:00.349126 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.348994    2580 scope.go:117] "RemoveContainer" containerID="d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"
Jul 18 01:51:00.349342 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.349306    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 18 01:51:00.349432 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.349350    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 18 01:51:00.349486 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:51:00.349346    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": container with ID starting with d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c not found: ID does not exist" containerID="d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"
Jul 18 01:51:00.349550 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.349482    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"} err="failed to get container status \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": rpc error: code = NotFound desc = could not find container \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": container with ID starting with d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c not found: ID does not exist"
Jul 18 01:51:00.349550 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.349505    2580 scope.go:117] "RemoveContainer" containerID="cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"
Jul 18 01:51:00.349703 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.349686    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 18 01:51:00.349771 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.349707    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 18 01:51:00.349877 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:51:00.349858    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": container with ID starting with cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903 not found: ID does not exist" containerID="cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"
Jul 18 01:51:00.349936 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.349883    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"} err="failed to get container status \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": rpc error: code = NotFound desc = could not find container \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": container with ID starting with cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903 not found: ID does not exist"
Jul 18 01:51:00.349999 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.349939    2580 scope.go:117] "RemoveContainer" containerID="f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"
Jul 18 01:51:00.349999 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.349976    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 18 01:51:00.350112 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350083    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 18 01:51:00.350112 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350093    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5klgan7bk4mm8\""
Jul 18 01:51:00.350207 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350146    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xdgtp\""
Jul 18 01:51:00.350307 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:51:00.350284    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": container with ID starting with f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6 not found: ID does not exist" containerID="f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"
Jul 18 01:51:00.350361 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350316    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"} err="failed to get container status \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": rpc error: code = NotFound desc = could not find container \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": container with ID starting with f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6 not found: ID does not exist"
Jul 18 01:51:00.350361 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350338    2580 scope.go:117] "RemoveContainer" containerID="148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"
Jul 18 01:51:00.350620 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350600    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 18 01:51:00.350702 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350609    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 18 01:51:00.350702 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:51:00.350682    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": container with ID starting with 148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787 not found: ID does not exist" containerID="148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"
Jul 18 01:51:00.350804 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350712    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"} err="failed to get container status \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": rpc error: code = NotFound desc = could not find container \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": container with ID starting with 148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787 not found: ID does not exist"
Jul 18 01:51:00.350804 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350729    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 18 01:51:00.350804 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350738    2580 scope.go:117] "RemoveContainer" containerID="3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"
Jul 18 01:51:00.350804 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350763    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 18 01:51:00.350804 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.350787    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 18 01:51:00.351033 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:51:00.351007    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": container with ID starting with 3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d not found: ID does not exist" containerID="3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"
Jul 18 01:51:00.351113 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.351037    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"} err="failed to get container status \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": rpc error: code = NotFound desc = could not find container \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": container with ID starting with 3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d not found: ID does not exist"
Jul 18 01:51:00.351113 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.351081    2580 scope.go:117] "RemoveContainer" containerID="a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"
Jul 18 01:51:00.351389 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:51:00.351360    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": container with ID starting with a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302 not found: ID does not exist" containerID="a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"
Jul 18 01:51:00.351491 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.351391    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"} err="failed to get container status \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": rpc error: code = NotFound desc = could not find container \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": container with ID starting with a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302 not found: ID does not exist"
Jul 18 01:51:00.351491 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.351406    2580 scope.go:117] "RemoveContainer" containerID="e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"
Jul 18 01:51:00.351678 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:51:00.351659    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": container with ID starting with e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a not found: ID does not exist" containerID="e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"
Jul 18 01:51:00.351743 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.351686    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"} err="failed to get container status \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": rpc error: code = NotFound desc = could not find container \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": container with ID starting with e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a not found: ID does not exist"
Jul 18 01:51:00.351743 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.351707    2580 scope.go:117] "RemoveContainer" containerID="d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"
Jul 18 01:51:00.351975 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.351948    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"} err="failed to get container status \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": rpc error: code = NotFound desc = could not find container \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": container with ID starting with d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c not found: ID does not exist"
Jul 18 01:51:00.351975 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.351973    2580 scope.go:117] "RemoveContainer" containerID="cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"
Jul 18 01:51:00.352240 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.352217    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"} err="failed to get container status \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": rpc error: code = NotFound desc = could not find container \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": container with ID starting with cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903 not found: ID does not exist"
Jul 18 01:51:00.352240 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.352240    2580 scope.go:117] "RemoveContainer" containerID="f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"
Jul 18 01:51:00.352530 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.352509    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"} err="failed to get container status \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": rpc error: code = NotFound desc = could not find container \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": container with ID starting with f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6 not found: ID does not exist"
Jul 18 01:51:00.352530 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.352530    2580 scope.go:117] "RemoveContainer" containerID="148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"
Jul 18 01:51:00.352649 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.352547    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 18 01:51:00.352755 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.352728    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"} err="failed to get container status \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": rpc error: code = NotFound desc = could not find container \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": container with ID starting with 148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787 not found: ID does not exist"
Jul 18 01:51:00.352850 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.352756    2580 scope.go:117] "RemoveContainer" containerID="3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"
Jul 18 01:51:00.353088 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.353012    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"} err="failed to get container status \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": rpc error: code = NotFound desc = could not find container \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": container with ID starting with 3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d not found: ID does not exist"
Jul 18 01:51:00.353088 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.353065    2580 scope.go:117] "RemoveContainer" containerID="a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"
Jul 18 01:51:00.353477 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.353444    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"} err="failed to get container status \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": rpc error: code = NotFound desc = could not find container \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": container with ID starting with a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302 not found: ID does not exist"
Jul 18 01:51:00.353477 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.353474    2580 scope.go:117] "RemoveContainer" containerID="e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"
Jul 18 01:51:00.354202 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.354176    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"} err="failed to get container status \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": rpc error: code = NotFound desc = could not find container \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": container with ID starting with e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a not found: ID does not exist"
Jul 18 01:51:00.354332 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.354203    2580 scope.go:117] "RemoveContainer" containerID="d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"
Jul 18 01:51:00.354505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.354465    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"} err="failed to get container status \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": rpc error: code = NotFound desc = could not find container \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": container with ID starting with d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c not found: ID does not exist"
Jul 18 01:51:00.354505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.354491    2580 scope.go:117] "RemoveContainer" containerID="cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"
Jul 18 01:51:00.354781 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.354757    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"} err="failed to get container status \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": rpc error: code = NotFound desc = could not find container \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": container with ID starting with cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903 not found: ID does not exist"
Jul 18 01:51:00.354781 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.354781    2580 scope.go:117] "RemoveContainer" containerID="f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"
Jul 18 01:51:00.355170 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.355141    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"} err="failed to get container status \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": rpc error: code = NotFound desc = could not find container \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": container with ID starting with f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6 not found: ID does not exist"
Jul 18 01:51:00.355254 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.355173    2580 scope.go:117] "RemoveContainer" containerID="148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"
Jul 18 01:51:00.355408 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.355381    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"} err="failed to get container status \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": rpc error: code = NotFound desc = could not find container \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": container with ID starting with 148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787 not found: ID does not exist"
Jul 18 01:51:00.355448 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.355413    2580 scope.go:117] "RemoveContainer" containerID="3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"
Jul 18 01:51:00.355482 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.355393    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 18 01:51:00.355660 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.355638    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"} err="failed to get container status \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": rpc error: code = NotFound desc = could not find container \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": container with ID starting with 3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d not found: ID does not exist"
Jul 18 01:51:00.355699 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.355664    2580 scope.go:117] "RemoveContainer" containerID="a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"
Jul 18 01:51:00.355895 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.355879    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"} err="failed to get container status \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": rpc error: code = NotFound desc = could not find container \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": container with ID starting with a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302 not found: ID does not exist"
Jul 18 01:51:00.355942 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.355897    2580 scope.go:117] "RemoveContainer" containerID="e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"
Jul 18 01:51:00.356131 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.356107    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"} err="failed to get container status \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": rpc error: code = NotFound desc = could not find container \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": container with ID starting with e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a not found: ID does not exist"
Jul 18 01:51:00.356184 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.356135    2580 scope.go:117] "RemoveContainer" containerID="d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"
Jul 18 01:51:00.356391 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.356373    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"} err="failed to get container status \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": rpc error: code = NotFound desc = could not find container \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": container with ID starting with d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c not found: ID does not exist"
Jul 18 01:51:00.356451 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.356393    2580 scope.go:117] "RemoveContainer" containerID="cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"
Jul 18 01:51:00.356626 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.356607    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"} err="failed to get container status \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": rpc error: code = NotFound desc = could not find container \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": container with ID starting with cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903 not found: ID does not exist"
Jul 18 01:51:00.356681 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.356627    2580 scope.go:117] "RemoveContainer" containerID="f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"
Jul 18 01:51:00.356854 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.356838    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"} err="failed to get container status \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": rpc error: code = NotFound desc = could not find container \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": container with ID starting with f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6 not found: ID does not exist"
Jul 18 01:51:00.356907 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.356855    2580 scope.go:117] "RemoveContainer" containerID="148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"
Jul 18 01:51:00.357083 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357066    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"} err="failed to get container status \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": rpc error: code = NotFound desc = could not find container \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": container with ID starting with 148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787 not found: ID does not exist"
Jul 18 01:51:00.357130 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357085    2580 scope.go:117] "RemoveContainer" containerID="3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"
Jul 18 01:51:00.357307 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357290    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"} err="failed to get container status \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": rpc error: code = NotFound desc = could not find container \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": container with ID starting with 3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d not found: ID does not exist"
Jul 18 01:51:00.357345 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357308    2580 scope.go:117] "RemoveContainer" containerID="a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"
Jul 18 01:51:00.357535 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357514    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"} err="failed to get container status \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": rpc error: code = NotFound desc = could not find container \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": container with ID starting with a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302 not found: ID does not exist"
Jul 18 01:51:00.357535 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357535    2580 scope.go:117] "RemoveContainer" containerID="e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"
Jul 18 01:51:00.357745 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357730    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"} err="failed to get container status \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": rpc error: code = NotFound desc = could not find container \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": container with ID starting with e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a not found: ID does not exist"
Jul 18 01:51:00.357782 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357747    2580 scope.go:117] "RemoveContainer" containerID="d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"
Jul 18 01:51:00.357981 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357965    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"} err="failed to get container status \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": rpc error: code = NotFound desc = could not find container \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": container with ID starting with d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c not found: ID does not exist"
Jul 18 01:51:00.358023 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.357983    2580 scope.go:117] "RemoveContainer" containerID="cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"
Jul 18 01:51:00.358237 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.358221    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"} err="failed to get container status \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": rpc error: code = NotFound desc = could not find container \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": container with ID starting with cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903 not found: ID does not exist"
Jul 18 01:51:00.358274 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.358238    2580 scope.go:117] "RemoveContainer" containerID="f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"
Jul 18 01:51:00.358461 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.358443    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"} err="failed to get container status \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": rpc error: code = NotFound desc = could not find container \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": container with ID starting with f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6 not found: ID does not exist"
Jul 18 01:51:00.358496 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.358461    2580 scope.go:117] "RemoveContainer" containerID="148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"
Jul 18 01:51:00.358665 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.358649    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"} err="failed to get container status \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": rpc error: code = NotFound desc = could not find container \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": container with ID starting with 148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787 not found: ID does not exist"
Jul 18 01:51:00.358711 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.358666    2580 scope.go:117] "RemoveContainer" containerID="3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"
Jul 18 01:51:00.358870 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.358848    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"} err="failed to get container status \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": rpc error: code = NotFound desc = could not find container \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": container with ID starting with 3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d not found: ID does not exist"
Jul 18 01:51:00.358870 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.358868    2580 scope.go:117] "RemoveContainer" containerID="a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"
Jul 18 01:51:00.359096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.359080    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"} err="failed to get container status \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": rpc error: code = NotFound desc = could not find container \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": container with ID starting with a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302 not found: ID does not exist"
Jul 18 01:51:00.359148 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.359097    2580 scope.go:117] "RemoveContainer" containerID="e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"
Jul 18 01:51:00.359300 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.359282    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"} err="failed to get container status \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": rpc error: code = NotFound desc = could not find container \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": container with ID starting with e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a not found: ID does not exist"
Jul 18 01:51:00.359342 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.359301    2580 scope.go:117] "RemoveContainer" containerID="d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"
Jul 18 01:51:00.359497 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.359480    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c"} err="failed to get container status \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": rpc error: code = NotFound desc = could not find container \"d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c\": container with ID starting with d231e9051785fe32bf955b06e2823cffb247e33a4edcfdd400ee32ac0a637a4c not found: ID does not exist"
Jul 18 01:51:00.359548 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.359497    2580 scope.go:117] "RemoveContainer" containerID="cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"
Jul 18 01:51:00.359740 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.359666    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903"} err="failed to get container status \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": rpc error: code = NotFound desc = could not find container \"cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903\": container with ID starting with cd1785d31eaa931c584846b092bf5470e405ff0555943c26e9e16b74a5f7c903 not found: ID does not exist"
Jul 18 01:51:00.359740 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.359697    2580 scope.go:117] "RemoveContainer" containerID="f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"
Jul 18 01:51:00.360074 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.360006    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6"} err="failed to get container status \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": rpc error: code = NotFound desc = could not find container \"f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6\": container with ID starting with f5e67e8e63c356a83fb2837bebadafe37511004feaad16bd9dfd12c100982ac6 not found: ID does not exist"
Jul 18 01:51:00.360074 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.360024    2580 scope.go:117] "RemoveContainer" containerID="148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"
Jul 18 01:51:00.360433 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.360407    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787"} err="failed to get container status \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": rpc error: code = NotFound desc = could not find container \"148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787\": container with ID starting with 148007e3e5a3a84a6b28fa911a28c37c408d7254686e3a00eb49bd2bd9046787 not found: ID does not exist"
Jul 18 01:51:00.360433 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.360434    2580 scope.go:117] "RemoveContainer" containerID="3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"
Jul 18 01:51:00.360693 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.360675    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d"} err="failed to get container status \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": rpc error: code = NotFound desc = could not find container \"3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d\": container with ID starting with 3e2c2bb42c2f8a29d64082ad0073d5248311458190e79847ca90483f0ba6fb4d not found: ID does not exist"
Jul 18 01:51:00.360693 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.360693    2580 scope.go:117] "RemoveContainer" containerID="a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"
Jul 18 01:51:00.360953 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.360932    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302"} err="failed to get container status \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": rpc error: code = NotFound desc = could not find container \"a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302\": container with ID starting with a014dc30769f59bf8b104c38703a026e03415ec531e089e33a482ae852ea2302 not found: ID does not exist"
Jul 18 01:51:00.360953 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.360952    2580 scope.go:117] "RemoveContainer" containerID="e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"
Jul 18 01:51:00.361219 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.361193    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a"} err="failed to get container status \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": rpc error: code = NotFound desc = could not find container \"e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a\": container with ID starting with e99e66a96247ed5ba47f6644b33099cf9f2f22a35a56dfffb576b00bc0b22b4a not found: ID does not exist"
Jul 18 01:51:00.362144 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.362123    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:51:00.369848 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.369823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.369939 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.369862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.370012 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.369941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.370095 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.370008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.370095 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.370056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.370202 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.370118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.370280 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.370260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/775e83c2-ae5f-4389-8198-55ed9a7921ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372152 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372337    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-config\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/775e83c2-ae5f-4389-8198-55ed9a7921ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372495    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvpvl\" (UniqueName: \"kubernetes.io/projected/775e83c2-ae5f-4389-8198-55ed9a7921ae-kube-api-access-pvpvl\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.372700 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.372651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.473896 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.473846    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.473896 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.473896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.473925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvpvl\" (UniqueName: \"kubernetes.io/projected/775e83c2-ae5f-4389-8198-55ed9a7921ae-kube-api-access-pvpvl\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.473967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.473994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/775e83c2-ae5f-4389-8198-55ed9a7921ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474259 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474845 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474845 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474845 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474845 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474845 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-config\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474845 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474845 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/775e83c2-ae5f-4389-8198-55ed9a7921ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474845 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.474845 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.474806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.475483 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.475297    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.477430 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.477264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.477430 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.477337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.477635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.477514    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/775e83c2-ae5f-4389-8198-55ed9a7921ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.477635 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.477537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.478404 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.477814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.478404 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.477919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.478404 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.478028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.478404 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.478198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.478404 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.478334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.478764 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.478443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.478764 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.478496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.479179 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.479156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.480096 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.480072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-config\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.480250 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.480234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/775e83c2-ae5f-4389-8198-55ed9a7921ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.480287 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.480271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/775e83c2-ae5f-4389-8198-55ed9a7921ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.481181 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.481160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/775e83c2-ae5f-4389-8198-55ed9a7921ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.482717 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.482693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvpvl\" (UniqueName: \"kubernetes.io/projected/775e83c2-ae5f-4389-8198-55ed9a7921ae-kube-api-access-pvpvl\") pod \"prometheus-k8s-0\" (UID: \"775e83c2-ae5f-4389-8198-55ed9a7921ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.657435 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.657342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:00.797871 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:00.797836    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:51:00.801153 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:51:00.801127    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775e83c2_ae5f_4389_8198_55ed9a7921ae.slice/crio-bda1d9019f1978ab7df8dda1aa7411531c3e7944287db07cb7b034930f57adc3 WatchSource:0}: Error finding container bda1d9019f1978ab7df8dda1aa7411531c3e7944287db07cb7b034930f57adc3: Status 404 returned error can't find the container with id bda1d9019f1978ab7df8dda1aa7411531c3e7944287db07cb7b034930f57adc3
Jul 18 01:51:01.291243 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:01.291211    2580 generic.go:356] "Generic (PLEG): container finished" podID="775e83c2-ae5f-4389-8198-55ed9a7921ae" containerID="2c023bacb045732a1da8f74eea7822a22a7e7e4c1f5e8bbbf0699db7b671f6bd" exitCode=0
Jul 18 01:51:01.291638 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:01.291294    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"775e83c2-ae5f-4389-8198-55ed9a7921ae","Type":"ContainerDied","Data":"2c023bacb045732a1da8f74eea7822a22a7e7e4c1f5e8bbbf0699db7b671f6bd"}
Jul 18 01:51:01.291638 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:01.291376    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"775e83c2-ae5f-4389-8198-55ed9a7921ae","Type":"ContainerStarted","Data":"bda1d9019f1978ab7df8dda1aa7411531c3e7944287db07cb7b034930f57adc3"}
Jul 18 01:51:01.390363 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:01.390327    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f708533-85c5-4b80-9fd6-fe01d167da3d" path="/var/lib/kubelet/pods/2f708533-85c5-4b80-9fd6-fe01d167da3d/volumes"
Jul 18 01:51:02.298765 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:02.298725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"775e83c2-ae5f-4389-8198-55ed9a7921ae","Type":"ContainerStarted","Data":"0a706833d9486eb790b58b99249d90f3a4276f1805e95ac9ac4aae2ef889ce45"}
Jul 18 01:51:02.299242 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:02.298798    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"775e83c2-ae5f-4389-8198-55ed9a7921ae","Type":"ContainerStarted","Data":"6ebb1de779a94c2718fbb43653259674e7abdc0b6eb36ac0fc721828c59a76c7"}
Jul 18 01:51:02.299242 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:02.298826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"775e83c2-ae5f-4389-8198-55ed9a7921ae","Type":"ContainerStarted","Data":"b9b827bd5f7900f1b0207c3b86fff5d6fc54f843b820142e89016cefab33b6d2"}
Jul 18 01:51:02.299242 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:02.298849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"775e83c2-ae5f-4389-8198-55ed9a7921ae","Type":"ContainerStarted","Data":"f6a0c69ec11c5d22539c51d879abe0cc75e0ec9f55576fa0169010549c154a48"}
Jul 18 01:51:02.299242 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:02.298872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"775e83c2-ae5f-4389-8198-55ed9a7921ae","Type":"ContainerStarted","Data":"05de247eee40ad9d578666706ee6557d9a996dff8c218ff679379627bdc3c2b1"}
Jul 18 01:51:02.299242 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:02.298897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"775e83c2-ae5f-4389-8198-55ed9a7921ae","Type":"ContainerStarted","Data":"e305fcee9bd51e5aa290e402a58893ab43c90d109b49a3bf018c8e3d6d824889"}
Jul 18 01:51:02.328942 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:02.328882    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.310650582 podStartE2EDuration="2.3288607s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.210118ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:51:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:02.327259808 +0000 UTC m=+155.536974052" watchObservedRunningTime="2026-07-18 01:51:02.3288607 +0000 UTC m=+155.538574921"
Jul 18 01:51:04.321875 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:04.321844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:04.420690 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:04.420659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:04.522204 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:04.522174    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:05.064382 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:05.064348    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:05.119061 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:05.119011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:05.658343 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:05.658310    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:51:14.914735 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:14.914698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:15.015726 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:15.015692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:15.074905 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:15.074878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:47.424953 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:47.424912    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:47.473187 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:51:47.473152    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:00.657503 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:00.657467    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:52:00.673153 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:00.673129    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:52:01.539030 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:01.539005    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:52:09.420233 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:09.420194    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:09.485575 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:09.485537    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:16.731683 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:16.731646    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:16.790649 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:16.790612    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:16.869987 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:16.869949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:16.932987 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:16.932958    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:17.466027 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:17.465991    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:17.535155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:17.535126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:21.368614 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:21.368562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:21.424502 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:21.424472    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:27.159260 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:27.159223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:27.220155 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:27.220124    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:27.273301 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:27.273271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:29.523732 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:29.523704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:29.625270 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:29.625237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:30.875984 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:30.875947    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:30.968494 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:30.968426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:36.423983 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:36.423953    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:48.007828 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:48.007734    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:48.106927 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:48.106895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:48.173776 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:48.173745    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:49.850071 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:49.850018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:50.395333 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:50.395298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:50.497062 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:52:50.497009    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:11.011753 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:11.011710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:17.400061 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:17.400008    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:19.324811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:19.324774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:21.012940 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:21.012903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:21.112372 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:21.112345    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:22.276340 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.276304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:22.343965 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.343935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:22.660127 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.660033    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:22.691698 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.691665    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-p286p"]
Jul 18 01:53:22.694923 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.694907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:22.697826 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.697803    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 18 01:53:22.698952 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.698934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-f9dkv\""
Jul 18 01:53:22.699076 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.698979    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 18 01:53:22.708698 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.708677    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-p286p"]
Jul 18 01:53:22.757530 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.757497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qlfp\" (UniqueName: \"kubernetes.io/projected/1390965f-77db-4874-a150-ef4d695f5341-kube-api-access-9qlfp\") pod \"cert-manager-webhook-6486ffd997-p286p\" (UID: \"1390965f-77db-4874-a150-ef4d695f5341\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:22.757675 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.757536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1390965f-77db-4874-a150-ef4d695f5341-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-p286p\" (UID: \"1390965f-77db-4874-a150-ef4d695f5341\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:22.772023 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.771999    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:22.858831 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.858796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1390965f-77db-4874-a150-ef4d695f5341-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-p286p\" (UID: \"1390965f-77db-4874-a150-ef4d695f5341\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:22.859028 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.858878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qlfp\" (UniqueName: \"kubernetes.io/projected/1390965f-77db-4874-a150-ef4d695f5341-kube-api-access-9qlfp\") pod \"cert-manager-webhook-6486ffd997-p286p\" (UID: \"1390965f-77db-4874-a150-ef4d695f5341\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:22.886566 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.886535    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1390965f-77db-4874-a150-ef4d695f5341-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-p286p\" (UID: \"1390965f-77db-4874-a150-ef4d695f5341\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:22.886694 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:22.886587    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qlfp\" (UniqueName: \"kubernetes.io/projected/1390965f-77db-4874-a150-ef4d695f5341-kube-api-access-9qlfp\") pod \"cert-manager-webhook-6486ffd997-p286p\" (UID: \"1390965f-77db-4874-a150-ef4d695f5341\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:23.017353 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:23.017324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:23.145024 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:23.144934    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-p286p"]
Jul 18 01:53:23.147757 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:53:23.147729    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1390965f_77db_4874_a150_ef4d695f5341.slice/crio-4fc3488520984ff4f164c78f462ccfe6d75a33ffdd1fb414a61d63b6e842d0b4 WatchSource:0}: Error finding container 4fc3488520984ff4f164c78f462ccfe6d75a33ffdd1fb414a61d63b6e842d0b4: Status 404 returned error can't find the container with id 4fc3488520984ff4f164c78f462ccfe6d75a33ffdd1fb414a61d63b6e842d0b4
Jul 18 01:53:23.829519 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:23.829488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-p286p" event={"ID":"1390965f-77db-4874-a150-ef4d695f5341","Type":"ContainerStarted","Data":"4fc3488520984ff4f164c78f462ccfe6d75a33ffdd1fb414a61d63b6e842d0b4"}
Jul 18 01:53:24.486250 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:24.486215    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:24.680151 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:24.680104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:24.910992 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:24.910823    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"]
Jul 18 01:53:24.914444 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:24.914414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"
Jul 18 01:53:24.917309 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:24.917206    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4wqnp\""
Jul 18 01:53:24.921513 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:24.921467    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"]
Jul 18 01:53:24.977546 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:24.977512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfw89\" (UniqueName: \"kubernetes.io/projected/4759ecba-fd0a-4ba0-b808-710820cad934-kube-api-access-nfw89\") pod \"cert-manager-cainjector-559cdd8c87-6r68n\" (UID: \"4759ecba-fd0a-4ba0-b808-710820cad934\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"
Jul 18 01:53:24.977736 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:24.977569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4759ecba-fd0a-4ba0-b808-710820cad934-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6r68n\" (UID: \"4759ecba-fd0a-4ba0-b808-710820cad934\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"
Jul 18 01:53:25.078898 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:25.078840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfw89\" (UniqueName: \"kubernetes.io/projected/4759ecba-fd0a-4ba0-b808-710820cad934-kube-api-access-nfw89\") pod \"cert-manager-cainjector-559cdd8c87-6r68n\" (UID: \"4759ecba-fd0a-4ba0-b808-710820cad934\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"
Jul 18 01:53:25.079105 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:25.078902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4759ecba-fd0a-4ba0-b808-710820cad934-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6r68n\" (UID: \"4759ecba-fd0a-4ba0-b808-710820cad934\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"
Jul 18 01:53:25.087850 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:25.087800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4759ecba-fd0a-4ba0-b808-710820cad934-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6r68n\" (UID: \"4759ecba-fd0a-4ba0-b808-710820cad934\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"
Jul 18 01:53:25.089066 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:25.089000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfw89\" (UniqueName: \"kubernetes.io/projected/4759ecba-fd0a-4ba0-b808-710820cad934-kube-api-access-nfw89\") pod \"cert-manager-cainjector-559cdd8c87-6r68n\" (UID: \"4759ecba-fd0a-4ba0-b808-710820cad934\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"
Jul 18 01:53:25.228640 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:25.228608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"
Jul 18 01:53:26.873595 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:26.873568    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6r68n"]
Jul 18 01:53:26.876990 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:53:26.876965    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4759ecba_fd0a_4ba0_b808_710820cad934.slice/crio-c7927af88bbe3782b77e73321ac268caec28ad51548a4d8260d15b7d5a748e67 WatchSource:0}: Error finding container c7927af88bbe3782b77e73321ac268caec28ad51548a4d8260d15b7d5a748e67: Status 404 returned error can't find the container with id c7927af88bbe3782b77e73321ac268caec28ad51548a4d8260d15b7d5a748e67
Jul 18 01:53:27.216793 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:27.216766    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 18 01:53:27.853177 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:27.853129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n" event={"ID":"4759ecba-fd0a-4ba0-b808-710820cad934","Type":"ContainerStarted","Data":"abae5d05e8d49f0189ca808ccdbe0b227d912673e05aed2e71746a88408d7e54"}
Jul 18 01:53:27.855353 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:27.853197    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n" event={"ID":"4759ecba-fd0a-4ba0-b808-710820cad934","Type":"ContainerStarted","Data":"c7927af88bbe3782b77e73321ac268caec28ad51548a4d8260d15b7d5a748e67"}
Jul 18 01:53:27.855353 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:27.854915    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-p286p" event={"ID":"1390965f-77db-4874-a150-ef4d695f5341","Type":"ContainerStarted","Data":"5943a8dc9c0d74463dc3f527f405a3dce14dd6faf3d6757fb8d0f2f80c033e4f"}
Jul 18 01:53:27.855353 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:27.854972    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:27.862738 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:27.862715    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:27.872515 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:27.872475    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6r68n" podStartSLOduration=3.8724622159999997 podStartE2EDuration="3.872462216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:27.872138565 +0000 UTC m=+301.081852796" watchObservedRunningTime="2026-07-18 01:53:27.872462216 +0000 UTC m=+301.082176437"
Jul 18 01:53:27.898796 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:27.898745    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-p286p" podStartSLOduration=2.238822967 podStartE2EDuration="5.898727178s" totalImagesPullingTime="3.659904211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:27.898103491 +0000 UTC m=+301.107817712" watchObservedRunningTime="2026-07-18 01:53:27.898727178 +0000 UTC m=+301.108441399"
Jul 18 01:53:27.971009 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:27.970972    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:32.890763 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:32.890734    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:33.861195 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:33.861168    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-p286p"
Jul 18 01:53:34.260989 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.260950    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lwp95"]
Jul 18 01:53:34.264202 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.264184    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lwp95"
Jul 18 01:53:34.266772 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.266751    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-6x9fv\""
Jul 18 01:53:34.271342 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.271271    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lwp95"]
Jul 18 01:53:34.356580 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.356552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbmr9\" (UniqueName: \"kubernetes.io/projected/3114e015-9d05-4c41-bb3d-36709ca8d62a-kube-api-access-xbmr9\") pod \"cert-manager-55b4d954d5-lwp95\" (UID: \"3114e015-9d05-4c41-bb3d-36709ca8d62a\") " pod="cert-manager/cert-manager-55b4d954d5-lwp95"
Jul 18 01:53:34.356742 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.356585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3114e015-9d05-4c41-bb3d-36709ca8d62a-bound-sa-token\") pod \"cert-manager-55b4d954d5-lwp95\" (UID: \"3114e015-9d05-4c41-bb3d-36709ca8d62a\") " pod="cert-manager/cert-manager-55b4d954d5-lwp95"
Jul 18 01:53:34.457871 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.457834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbmr9\" (UniqueName: \"kubernetes.io/projected/3114e015-9d05-4c41-bb3d-36709ca8d62a-kube-api-access-xbmr9\") pod \"cert-manager-55b4d954d5-lwp95\" (UID: \"3114e015-9d05-4c41-bb3d-36709ca8d62a\") " pod="cert-manager/cert-manager-55b4d954d5-lwp95"
Jul 18 01:53:34.457871 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.457875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3114e015-9d05-4c41-bb3d-36709ca8d62a-bound-sa-token\") pod \"cert-manager-55b4d954d5-lwp95\" (UID: \"3114e015-9d05-4c41-bb3d-36709ca8d62a\") " pod="cert-manager/cert-manager-55b4d954d5-lwp95"
Jul 18 01:53:34.467062 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.467013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3114e015-9d05-4c41-bb3d-36709ca8d62a-bound-sa-token\") pod \"cert-manager-55b4d954d5-lwp95\" (UID: \"3114e015-9d05-4c41-bb3d-36709ca8d62a\") " pod="cert-manager/cert-manager-55b4d954d5-lwp95"
Jul 18 01:53:34.467241 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.467222    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbmr9\" (UniqueName: \"kubernetes.io/projected/3114e015-9d05-4c41-bb3d-36709ca8d62a-kube-api-access-xbmr9\") pod \"cert-manager-55b4d954d5-lwp95\" (UID: \"3114e015-9d05-4c41-bb3d-36709ca8d62a\") " pod="cert-manager/cert-manager-55b4d954d5-lwp95"
Jul 18 01:53:34.574315 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.574248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lwp95"
Jul 18 01:53:34.700091 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.700062    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lwp95"]
Jul 18 01:53:34.703464 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:53:34.703434    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3114e015_9d05_4c41_bb3d_36709ca8d62a.slice/crio-23104370a1e6fe7df43c14f49b5a245b6f86118f5b2b9374df88f348659fd585 WatchSource:0}: Error finding container 23104370a1e6fe7df43c14f49b5a245b6f86118f5b2b9374df88f348659fd585: Status 404 returned error can't find the container with id 23104370a1e6fe7df43c14f49b5a245b6f86118f5b2b9374df88f348659fd585
Jul 18 01:53:34.884147 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.884071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lwp95" event={"ID":"3114e015-9d05-4c41-bb3d-36709ca8d62a","Type":"ContainerStarted","Data":"76edb3b1fe7dc2b67c348f4ee25198d9d3e3767b1eb7ed6b3fdc3eca3e54c8b1"}
Jul 18 01:53:34.884147 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.884111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lwp95" event={"ID":"3114e015-9d05-4c41-bb3d-36709ca8d62a","Type":"ContainerStarted","Data":"23104370a1e6fe7df43c14f49b5a245b6f86118f5b2b9374df88f348659fd585"}
Jul 18 01:53:34.899419 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:34.899374    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lwp95" podStartSLOduration=0.899358988 podStartE2EDuration="899.358988ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:34.898883935 +0000 UTC m=+308.108598156" watchObservedRunningTime="2026-07-18 01:53:34.899358988 +0000 UTC m=+308.109073208"
Jul 18 01:53:35.363151 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:35.363118    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:35.460833 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:35.460803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:39.954299 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:39.954266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:40.061515 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:40.061478    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:42.107802 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:42.107771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:47.053628 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:47.053589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:47.387430 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:47.387344    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:57.460725 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:57.460695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:57.557099 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:57.557067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:58.190300 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.190262    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:58.577148 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.577073    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:58.659316 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.659284    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"]
Jul 18 01:53:58.662526 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.662509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.667268 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.667249    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 18 01:53:58.667655 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.667638    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 18 01:53:58.668037 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.668016    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:53:58.668283 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.668260    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dfvpq\""
Jul 18 01:53:58.668419 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.668264    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 18 01:53:58.668419 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.668354    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 18 01:53:58.678979 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.678949    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"]
Jul 18 01:53:58.737314 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.737284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:58.757302 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.757277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0b9531fb-63b3-44b5-ac95-2e5e431ac240-manager-config\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.757418 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.757318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b9531fb-63b3-44b5-ac95-2e5e431ac240-cert\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.757418 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.757339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b9531fb-63b3-44b5-ac95-2e5e431ac240-metrics-cert\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.757418 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.757407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdvsh\" (UniqueName: \"kubernetes.io/projected/0b9531fb-63b3-44b5-ac95-2e5e431ac240-kube-api-access-fdvsh\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.858108 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.858011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0b9531fb-63b3-44b5-ac95-2e5e431ac240-manager-config\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.858288 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.858124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b9531fb-63b3-44b5-ac95-2e5e431ac240-cert\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.858288 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.858166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b9531fb-63b3-44b5-ac95-2e5e431ac240-metrics-cert\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.858288 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.858206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdvsh\" (UniqueName: \"kubernetes.io/projected/0b9531fb-63b3-44b5-ac95-2e5e431ac240-kube-api-access-fdvsh\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.859143 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.858893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0b9531fb-63b3-44b5-ac95-2e5e431ac240-manager-config\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.860982 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.860953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b9531fb-63b3-44b5-ac95-2e5e431ac240-cert\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.861124 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.861033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b9531fb-63b3-44b5-ac95-2e5e431ac240-metrics-cert\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.872848 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.872817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdvsh\" (UniqueName: \"kubernetes.io/projected/0b9531fb-63b3-44b5-ac95-2e5e431ac240-kube-api-access-fdvsh\") pod \"lws-controller-manager-6bc78dbd6c-fpqjb\" (UID: \"0b9531fb-63b3-44b5-ac95-2e5e431ac240\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:58.971924 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:58.971879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:53:59.057878 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.057846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:59.148449 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.148423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"]
Jul 18 01:53:59.150204 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:53:59.150181    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9531fb_63b3_44b5_ac95_2e5e431ac240.slice/crio-68378a6c62033f0fe6eb4b8ec6b5f7d8da7787b3480cf7c0096d1953d5be9860 WatchSource:0}: Error finding container 68378a6c62033f0fe6eb4b8ec6b5f7d8da7787b3480cf7c0096d1953d5be9860: Status 404 returned error can't find the container with id 68378a6c62033f0fe6eb4b8ec6b5f7d8da7787b3480cf7c0096d1953d5be9860
Jul 18 01:53:59.152100 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.152083    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:53:59.225290 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.225238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:59.295183 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.295149    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"]
Jul 18 01:53:59.299203 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.299182    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.302358 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.302336    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hw5p9\""
Jul 18 01:53:59.302517 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.302500    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 18 01:53:59.302748 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.302734    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 18 01:53:59.302902 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.302884    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 18 01:53:59.303151 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.303137    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 18 01:53:59.354390 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.354357    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"]
Jul 18 01:53:59.362817 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.362793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d0b35551-ad37-49f0-aa8f-f94340c1828e-manifests\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.362945 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.362826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d0b35551-ad37-49f0-aa8f-f94340c1828e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.362945 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.362854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0b35551-ad37-49f0-aa8f-f94340c1828e-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.362945 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.362876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd6xm\" (UniqueName: \"kubernetes.io/projected/d0b35551-ad37-49f0-aa8f-f94340c1828e-kube-api-access-gd6xm\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.363091 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.362988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0b35551-ad37-49f0-aa8f-f94340c1828e-tmp\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.464055 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.464016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0b35551-ad37-49f0-aa8f-f94340c1828e-tmp\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.464256 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.464081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d0b35551-ad37-49f0-aa8f-f94340c1828e-manifests\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.464256 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.464116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d0b35551-ad37-49f0-aa8f-f94340c1828e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.464256 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.464135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0b35551-ad37-49f0-aa8f-f94340c1828e-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.464256 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.464151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd6xm\" (UniqueName: \"kubernetes.io/projected/d0b35551-ad37-49f0-aa8f-f94340c1828e-kube-api-access-gd6xm\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.464695 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.464672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d0b35551-ad37-49f0-aa8f-f94340c1828e-manifests\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.466479 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.466460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0b35551-ad37-49f0-aa8f-f94340c1828e-tmp\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.466565 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.466550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d0b35551-ad37-49f0-aa8f-f94340c1828e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.466604 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.466581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0b35551-ad37-49f0-aa8f-f94340c1828e-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.491970 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.491948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd6xm\" (UniqueName: \"kubernetes.io/projected/d0b35551-ad37-49f0-aa8f-f94340c1828e-kube-api-access-gd6xm\") pod \"opendatahub-operator-controller-manager-84c588c4c7-lbflf\" (UID: \"d0b35551-ad37-49f0-aa8f-f94340c1828e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.610555 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.610518    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:53:59.629180 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.629148    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:59.717206 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.717107    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:59.832644 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.832559    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"]
Jul 18 01:53:59.838154 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:53:59.838124    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b35551_ad37_49f0_aa8f_f94340c1828e.slice/crio-997a35eff70fa7349abbf09a27266e3822d281c5c6302b21f34d66368e49d009 WatchSource:0}: Error finding container 997a35eff70fa7349abbf09a27266e3822d281c5c6302b21f34d66368e49d009: Status 404 returned error can't find the container with id 997a35eff70fa7349abbf09a27266e3822d281c5c6302b21f34d66368e49d009
Jul 18 01:53:59.928474 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.928435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:59.986331 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.986184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb" event={"ID":"0b9531fb-63b3-44b5-ac95-2e5e431ac240","Type":"ContainerStarted","Data":"68378a6c62033f0fe6eb4b8ec6b5f7d8da7787b3480cf7c0096d1953d5be9860"}
Jul 18 01:53:59.988397 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:53:59.988368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf" event={"ID":"d0b35551-ad37-49f0-aa8f-f94340c1828e","Type":"ContainerStarted","Data":"997a35eff70fa7349abbf09a27266e3822d281c5c6302b21f34d66368e49d009"}
Jul 18 01:54:00.026360 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:00.026314    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:00.164670 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:00.164629    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:00.311833 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:00.311753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:04.010751 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:04.010717    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb" event={"ID":"0b9531fb-63b3-44b5-ac95-2e5e431ac240","Type":"ContainerStarted","Data":"94885c813bbac444d4725692a579371df1861dc38d8019ead835cb91be8d4b1e"}
Jul 18 01:54:04.011255 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:04.010797    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:54:04.012243 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:04.012214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf" event={"ID":"d0b35551-ad37-49f0-aa8f-f94340c1828e","Type":"ContainerStarted","Data":"43efa4b21ca75c9c1602153d65477562e0f3910c2b0816bde7ee8c0a59b1fa90"}
Jul 18 01:54:04.099217 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:04.099156    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb" podStartSLOduration=1.8398530690000001 podStartE2EDuration="6.09913928s" totalImagesPullingTime="4.259286211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:04.047749171 +0000 UTC m=+337.257463391" watchObservedRunningTime="2026-07-18 01:54:04.09913928 +0000 UTC m=+337.308853500"
Jul 18 01:54:05.018921 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:05.018886    2580 generic.go:356] "Generic (PLEG): container finished" podID="d0b35551-ad37-49f0-aa8f-f94340c1828e" containerID="43efa4b21ca75c9c1602153d65477562e0f3910c2b0816bde7ee8c0a59b1fa90" exitCode=0
Jul 18 01:54:05.019346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:05.018977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf" event={"ID":"d0b35551-ad37-49f0-aa8f-f94340c1828e","Type":"ContainerDied","Data":"43efa4b21ca75c9c1602153d65477562e0f3910c2b0816bde7ee8c0a59b1fa90"}
Jul 18 01:54:06.024870 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:06.024830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf" event={"ID":"d0b35551-ad37-49f0-aa8f-f94340c1828e","Type":"ContainerStarted","Data":"a369d5a609b6d06878175f847644f65d6707da3e97372c644b1ace409725e5b3"}
Jul 18 01:54:06.025294 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:06.024985    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:54:06.048864 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:06.048809    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf" podStartSLOduration=2.032631017 podStartE2EDuration="7.04879766s" totalImagesPullingTime="3.888789277s" totalInitContainerRuntime="1.127377366s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:06.047320589 +0000 UTC m=+339.257034809" watchObservedRunningTime="2026-07-18 01:54:06.04879766 +0000 UTC m=+339.258511911"
Jul 18 01:54:15.021851 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:15.021823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-fpqjb"
Jul 18 01:54:17.032484 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:17.032455    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-lbflf"
Jul 18 01:54:18.610026 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:18.609993    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:18.714017 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:18.713988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:19.120457 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.120418    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:19.375634 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.375555    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"]
Jul 18 01:54:19.377916 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.377900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.381683 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.381661    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 18 01:54:19.381811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.381666    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 18 01:54:19.381811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.381742    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 18 01:54:19.381811 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.381773    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 18 01:54:19.381947 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.381822    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9ws6j\""
Jul 18 01:54:19.400432 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.400408    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"]
Jul 18 01:54:19.463470 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.463435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:19.542649 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.542621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qql46\" (UniqueName: \"kubernetes.io/projected/6e9d01ad-b24f-4469-852c-50b8f78ef8df-kube-api-access-qql46\") pod \"kube-auth-proxy-7c65d4b6f-nwxt2\" (UID: \"6e9d01ad-b24f-4469-852c-50b8f78ef8df\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.542807 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.542697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e9d01ad-b24f-4469-852c-50b8f78ef8df-tmp\") pod \"kube-auth-proxy-7c65d4b6f-nwxt2\" (UID: \"6e9d01ad-b24f-4469-852c-50b8f78ef8df\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.542807 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.542729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e9d01ad-b24f-4469-852c-50b8f78ef8df-tls-certs\") pod \"kube-auth-proxy-7c65d4b6f-nwxt2\" (UID: \"6e9d01ad-b24f-4469-852c-50b8f78ef8df\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.643228 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.643141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qql46\" (UniqueName: \"kubernetes.io/projected/6e9d01ad-b24f-4469-852c-50b8f78ef8df-kube-api-access-qql46\") pod \"kube-auth-proxy-7c65d4b6f-nwxt2\" (UID: \"6e9d01ad-b24f-4469-852c-50b8f78ef8df\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.643644 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.643238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e9d01ad-b24f-4469-852c-50b8f78ef8df-tmp\") pod \"kube-auth-proxy-7c65d4b6f-nwxt2\" (UID: \"6e9d01ad-b24f-4469-852c-50b8f78ef8df\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.643644 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.643262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e9d01ad-b24f-4469-852c-50b8f78ef8df-tls-certs\") pod \"kube-auth-proxy-7c65d4b6f-nwxt2\" (UID: \"6e9d01ad-b24f-4469-852c-50b8f78ef8df\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.645730 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.645704    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e9d01ad-b24f-4469-852c-50b8f78ef8df-tmp\") pod \"kube-auth-proxy-7c65d4b6f-nwxt2\" (UID: \"6e9d01ad-b24f-4469-852c-50b8f78ef8df\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.645894 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.645875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e9d01ad-b24f-4469-852c-50b8f78ef8df-tls-certs\") pod \"kube-auth-proxy-7c65d4b6f-nwxt2\" (UID: \"6e9d01ad-b24f-4469-852c-50b8f78ef8df\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.652026 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.651998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qql46\" (UniqueName: \"kubernetes.io/projected/6e9d01ad-b24f-4469-852c-50b8f78ef8df-kube-api-access-qql46\") pod \"kube-auth-proxy-7c65d4b6f-nwxt2\" (UID: \"6e9d01ad-b24f-4469-852c-50b8f78ef8df\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.687599 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.687575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"
Jul 18 01:54:19.707445 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.707390    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:19.809953 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.809921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:19.833946 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.833916    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2"]
Jul 18 01:54:19.842316 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:54:19.842284    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e9d01ad_b24f_4469_852c_50b8f78ef8df.slice/crio-f2bbd32cdb1b0d779073555b0643064bbe60d89b7821c3389c66aaf14233e661 WatchSource:0}: Error finding container f2bbd32cdb1b0d779073555b0643064bbe60d89b7821c3389c66aaf14233e661: Status 404 returned error can't find the container with id f2bbd32cdb1b0d779073555b0643064bbe60d89b7821c3389c66aaf14233e661
Jul 18 01:54:19.961613 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:19.961579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:20.094917 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:20.094876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2" event={"ID":"6e9d01ad-b24f-4469-852c-50b8f78ef8df","Type":"ContainerStarted","Data":"f2bbd32cdb1b0d779073555b0643064bbe60d89b7821c3389c66aaf14233e661"}
Jul 18 01:54:20.121026 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:20.120989    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:20.681195 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:20.681160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:20.843760 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:20.843730    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:20.935503 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:20.935423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:21.895247 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:21.895215    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:23.109618 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:23.109580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2" event={"ID":"6e9d01ad-b24f-4469-852c-50b8f78ef8df","Type":"ContainerStarted","Data":"7338b1c61d5752ae40b25d8710bf67893e8828718bbc3f185abfee0f21f6de88"}
Jul 18 01:54:23.126730 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:23.126682    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-nwxt2" podStartSLOduration=1.82517737 podStartE2EDuration="4.126666488s" totalImagesPullingTime="2.301489118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:23.12553572 +0000 UTC m=+356.335249941" watchObservedRunningTime="2026-07-18 01:54:23.126666488 +0000 UTC m=+356.336380707"
Jul 18 01:54:26.576603 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:26.576563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:26.650678 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:26.650645    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:27.002272 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:27.002241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:27.194425 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:27.194393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:27.295505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:27.295434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:30.843330 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:30.843293    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:31.115786 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:31.115704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:31.238630 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:31.238602    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:32.187012 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:32.186982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:32.457015 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:32.456933    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:36.263796 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:36.263764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:36.342879 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:36.342850    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:41.953539 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:54:41.953502    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:06.700741 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:06.700715    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:06.801677 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:06.801653    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:20.102808 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:20.102773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:20.209236 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:20.209201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:20.289188 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:20.289155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:21.017221 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:21.017187    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:21.281495 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:21.281412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:40.055997 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:40.055960    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:40.156134 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:40.156094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:49.775354 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:49.775277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:53.723227 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:53.723189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:56.154158 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:56.154116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:57.574289 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:57.574216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:58.916284 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:55:58.916218    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:01.156267 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:01.156218    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:01.354952 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:01.354909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:01.515626 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:01.515587    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:01.612291 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:01.612236    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:04.165657 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:04.165620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:05.212610 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:05.212582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:05.457649 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:05.457620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:05.694473 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:05.694434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:05.789498 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:05.789464    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:05.890666 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:05.890635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:06.436223 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:06.436194    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:06.555998 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:06.555966    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:09.307303 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:09.307266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:09.438923 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:09.438891    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:15.549004 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.548968    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"]
Jul 18 01:56:15.552457 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.552438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"
Jul 18 01:56:15.555525 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.555505    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 18 01:56:15.556722 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.556699    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6s6s7\""
Jul 18 01:56:15.556830 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.556725    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 18 01:56:15.561741 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.561719    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"]
Jul 18 01:56:15.737127 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.737085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4md6b\" (UniqueName: \"kubernetes.io/projected/79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2-kube-api-access-4md6b\") pod \"limitador-operator-controller-manager-859d7b78bd-gwqr7\" (UID: \"79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"
Jul 18 01:56:15.837833 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.837727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4md6b\" (UniqueName: \"kubernetes.io/projected/79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2-kube-api-access-4md6b\") pod \"limitador-operator-controller-manager-859d7b78bd-gwqr7\" (UID: \"79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"
Jul 18 01:56:15.847969 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.847936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4md6b\" (UniqueName: \"kubernetes.io/projected/79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2-kube-api-access-4md6b\") pod \"limitador-operator-controller-manager-859d7b78bd-gwqr7\" (UID: \"79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"
Jul 18 01:56:15.863471 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:15.863439    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"
Jul 18 01:56:16.002288 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:16.002148    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"]
Jul 18 01:56:16.004582 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:56:16.004545    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e8a9b7_fe08_4d3e_9bf6_19bd688a17c2.slice/crio-e72c22b89ef1957c5c1b28164b3b09cd17c7ef2821307658571de671162bad99 WatchSource:0}: Error finding container e72c22b89ef1957c5c1b28164b3b09cd17c7ef2821307658571de671162bad99: Status 404 returned error can't find the container with id e72c22b89ef1957c5c1b28164b3b09cd17c7ef2821307658571de671162bad99
Jul 18 01:56:16.255520 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:16.255475    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:16.361860 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:16.361827    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:16.456239 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:16.456208    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:16.557736 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:16.557654    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:16.601297 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:16.601267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7" event={"ID":"79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2","Type":"ContainerStarted","Data":"e72c22b89ef1957c5c1b28164b3b09cd17c7ef2821307658571de671162bad99"}
Jul 18 01:56:16.731664 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:16.731630    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:16.835685 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:16.835568    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:18.613123 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:18.613091    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7" event={"ID":"79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2","Type":"ContainerStarted","Data":"afb66ee7676ac8da843cb1e8eb9673b470729c27077ffb1a2a2ea3eee9e5e1d8"}
Jul 18 01:56:18.613510 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:18.613221    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"
Jul 18 01:56:18.630019 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:18.629967    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7" podStartSLOduration=1.980765791 podStartE2EDuration="3.629950576s" totalImagesPullingTime="1.649184785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:18.629232365 +0000 UTC m=+471.838946587" watchObservedRunningTime="2026-07-18 01:56:18.629950576 +0000 UTC m=+471.839664798"
Jul 18 01:56:21.528013 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:21.527975    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:21.631011 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:21.630973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:26.826196 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:26.826164    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:26.946794 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:26.946749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:29.619998 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:29.619967    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gwqr7"
Jul 18 01:56:30.722732 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:30.722695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:30.791998 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:30.791959    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:38.922305 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:38.922243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:39.026710 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:39.026668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:43.515547 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:43.515514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:43.573309 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:43.573278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:43.912675 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:43.912588    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:43.966754 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:43.966722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:55.358013 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:55.357977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:55.692383 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:56:55.692349    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:00.048240 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:00.048203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:00.131849 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:00.131818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:00.186248 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:00.186217    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:00.838003 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:00.837948    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:00.898153 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:00.898123    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:16.197143 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:16.197102    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:16.295533 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:16.295500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:16.672683 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:16.672644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:28.555760 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:28.555681    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:28.677657 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:28.677624    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:34.894228 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:34.894194    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:34.910553 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:34.910523    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-864ff85df7-4sbns"]
Jul 18 01:57:34.913861 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:34.913842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:34.916493 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:34.916465    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 01:57:34.916493 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:34.916479    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 01:57:34.916651 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:34.916496    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zpvtf\""
Jul 18 01:57:34.928505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:34.926200    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-864ff85df7-4sbns"]
Jul 18 01:57:34.964879 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:34.964853    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:35.022737 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.022713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:35.042917 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.042887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a08e10f-7034-45c6-9217-b3bcab46cfe8-webhook-cert\") pod \"maas-controller-864ff85df7-4sbns\" (UID: \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\") " pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:35.043024 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.042930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpxvr\" (UniqueName: \"kubernetes.io/projected/1a08e10f-7034-45c6-9217-b3bcab46cfe8-kube-api-access-wpxvr\") pod \"maas-controller-864ff85df7-4sbns\" (UID: \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\") " pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:35.143353 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.143320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpxvr\" (UniqueName: \"kubernetes.io/projected/1a08e10f-7034-45c6-9217-b3bcab46cfe8-kube-api-access-wpxvr\") pod \"maas-controller-864ff85df7-4sbns\" (UID: \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\") " pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:35.143488 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.143419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a08e10f-7034-45c6-9217-b3bcab46cfe8-webhook-cert\") pod \"maas-controller-864ff85df7-4sbns\" (UID: \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\") " pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:35.143529 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:57:35.143520    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 18 01:57:35.143600 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:57:35.143590    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a08e10f-7034-45c6-9217-b3bcab46cfe8-webhook-cert podName:1a08e10f-7034-45c6-9217-b3bcab46cfe8 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:35.643573792 +0000 UTC m=+548.853287989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/1a08e10f-7034-45c6-9217-b3bcab46cfe8-webhook-cert") pod "maas-controller-864ff85df7-4sbns" (UID: "1a08e10f-7034-45c6-9217-b3bcab46cfe8") : secret "maas-controller-webhook-cert" not found
Jul 18 01:57:35.154108 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.154030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpxvr\" (UniqueName: \"kubernetes.io/projected/1a08e10f-7034-45c6-9217-b3bcab46cfe8-kube-api-access-wpxvr\") pod \"maas-controller-864ff85df7-4sbns\" (UID: \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\") " pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:35.266777 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.266742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:35.648137 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.648099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a08e10f-7034-45c6-9217-b3bcab46cfe8-webhook-cert\") pod \"maas-controller-864ff85df7-4sbns\" (UID: \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\") " pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:35.650733 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.650707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a08e10f-7034-45c6-9217-b3bcab46cfe8-webhook-cert\") pod \"maas-controller-864ff85df7-4sbns\" (UID: \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\") " pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:35.824371 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.824338    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:35.955979 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:35.955955    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-864ff85df7-4sbns"]
Jul 18 01:57:35.958402 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:57:35.958376    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a08e10f_7034_45c6_9217_b3bcab46cfe8.slice/crio-55e8b50b70f22e67e9b10b040b0e508b936f57ac145ba848fcb78d1e8d71cd1b WatchSource:0}: Error finding container 55e8b50b70f22e67e9b10b040b0e508b936f57ac145ba848fcb78d1e8d71cd1b: Status 404 returned error can't find the container with id 55e8b50b70f22e67e9b10b040b0e508b936f57ac145ba848fcb78d1e8d71cd1b
Jul 18 01:57:36.958607 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:36.958567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864ff85df7-4sbns" event={"ID":"1a08e10f-7034-45c6-9217-b3bcab46cfe8","Type":"ContainerStarted","Data":"55e8b50b70f22e67e9b10b040b0e508b936f57ac145ba848fcb78d1e8d71cd1b"}
Jul 18 01:57:38.447276 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:38.447237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:38.549827 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:38.549747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:38.968881 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:38.968840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864ff85df7-4sbns" event={"ID":"1a08e10f-7034-45c6-9217-b3bcab46cfe8","Type":"ContainerStarted","Data":"86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187"}
Jul 18 01:57:38.969082 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:38.968911    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:39.021184 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:39.021134    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-864ff85df7-4sbns" podStartSLOduration=2.726175594 podStartE2EDuration="5.021117489s" totalImagesPullingTime="2.294941895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:39.019809131 +0000 UTC m=+552.229523342" watchObservedRunningTime="2026-07-18 01:57:39.021117489 +0000 UTC m=+552.230831709"
Jul 18 01:57:49.975173 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:49.975139    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:50.360704 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:50.360614    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-864ff85df7-4sbns"]
Jul 18 01:57:50.360844 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:50.360824    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-864ff85df7-4sbns" podUID="1a08e10f-7034-45c6-9217-b3bcab46cfe8" containerName="manager" containerID="cri-o://86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187" gracePeriod=30
Jul 18 01:57:50.609346 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:50.609322    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:50.665807 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:50.665785    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a08e10f-7034-45c6-9217-b3bcab46cfe8-webhook-cert\") pod \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\" (UID: \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\") "
Jul 18 01:57:50.665970 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:50.665836    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpxvr\" (UniqueName: \"kubernetes.io/projected/1a08e10f-7034-45c6-9217-b3bcab46cfe8-kube-api-access-wpxvr\") pod \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\" (UID: \"1a08e10f-7034-45c6-9217-b3bcab46cfe8\") "
Jul 18 01:57:50.668069 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:50.668021    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a08e10f-7034-45c6-9217-b3bcab46cfe8-kube-api-access-wpxvr" (OuterVolumeSpecName: "kube-api-access-wpxvr") pod "1a08e10f-7034-45c6-9217-b3bcab46cfe8" (UID: "1a08e10f-7034-45c6-9217-b3bcab46cfe8"). InnerVolumeSpecName "kube-api-access-wpxvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:57:50.668180 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:50.668072    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a08e10f-7034-45c6-9217-b3bcab46cfe8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1a08e10f-7034-45c6-9217-b3bcab46cfe8" (UID: "1a08e10f-7034-45c6-9217-b3bcab46cfe8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:57:50.767098 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:50.767036    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a08e10f-7034-45c6-9217-b3bcab46cfe8-webhook-cert\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:57:50.767098 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:50.767096    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpxvr\" (UniqueName: \"kubernetes.io/projected/1a08e10f-7034-45c6-9217-b3bcab46cfe8-kube-api-access-wpxvr\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 01:57:51.023437 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.023348    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a08e10f-7034-45c6-9217-b3bcab46cfe8" containerID="86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187" exitCode=0
Jul 18 01:57:51.023437 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.023393    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864ff85df7-4sbns"
Jul 18 01:57:51.023437 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.023429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864ff85df7-4sbns" event={"ID":"1a08e10f-7034-45c6-9217-b3bcab46cfe8","Type":"ContainerDied","Data":"86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187"}
Jul 18 01:57:51.023933 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.023473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864ff85df7-4sbns" event={"ID":"1a08e10f-7034-45c6-9217-b3bcab46cfe8","Type":"ContainerDied","Data":"55e8b50b70f22e67e9b10b040b0e508b936f57ac145ba848fcb78d1e8d71cd1b"}
Jul 18 01:57:51.023933 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.023498    2580 scope.go:117] "RemoveContainer" containerID="86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187"
Jul 18 01:57:51.032332 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.032317    2580 scope.go:117] "RemoveContainer" containerID="86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187"
Jul 18 01:57:51.032569 ip-10-0-142-56 kubenswrapper[2580]: E0718 01:57:51.032552    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187\": container with ID starting with 86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187 not found: ID does not exist" containerID="86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187"
Jul 18 01:57:51.032624 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.032576    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187"} err="failed to get container status \"86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187\": rpc error: code = NotFound desc = could not find container \"86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187\": container with ID starting with 86d9a3f1a587094c132faa55dfd95df049e4a4a94ddb4229b859f1e6e4d2b187 not found: ID does not exist"
Jul 18 01:57:51.042584 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.042563    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-864ff85df7-4sbns"]
Jul 18 01:57:51.044613 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.044594    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-864ff85df7-4sbns"]
Jul 18 01:57:51.390745 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:57:51.390665    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a08e10f-7034-45c6-9217-b3bcab46cfe8" path="/var/lib/kubelet/pods/1a08e10f-7034-45c6-9217-b3bcab46cfe8/volumes"
Jul 18 01:58:10.364575 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:10.364542    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:10.467095 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:10.467061    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:10.827930 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:10.827897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:11.171893 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.171861    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:11.296503 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.296468    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:11.362851 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.362818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:11.400168 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.400136    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"]
Jul 18 01:58:11.400534 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.400469    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a08e10f-7034-45c6-9217-b3bcab46cfe8" containerName="manager"
Jul 18 01:58:11.400534 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.400478    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a08e10f-7034-45c6-9217-b3bcab46cfe8" containerName="manager"
Jul 18 01:58:11.400608 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.400535    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a08e10f-7034-45c6-9217-b3bcab46cfe8" containerName="manager"
Jul 18 01:58:11.404673 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.404653    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:11.408764 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.408743    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-n757b\""
Jul 18 01:58:11.408855 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.408742    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 18 01:58:11.416147 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.416126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"]
Jul 18 01:58:11.544742 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.544666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed1d004c-b99f-4c1c-b665-86e51c2d53fb-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wg7gq\" (UID: \"ed1d004c-b99f-4c1c-b665-86e51c2d53fb\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:11.544889 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.544745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9kwl\" (UniqueName: \"kubernetes.io/projected/ed1d004c-b99f-4c1c-b665-86e51c2d53fb-kube-api-access-c9kwl\") pod \"payload-pre-processing-7677b774bb-wg7gq\" (UID: \"ed1d004c-b99f-4c1c-b665-86e51c2d53fb\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:11.646080 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.646028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed1d004c-b99f-4c1c-b665-86e51c2d53fb-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wg7gq\" (UID: \"ed1d004c-b99f-4c1c-b665-86e51c2d53fb\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:11.646239 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.646168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9kwl\" (UniqueName: \"kubernetes.io/projected/ed1d004c-b99f-4c1c-b665-86e51c2d53fb-kube-api-access-c9kwl\") pod \"payload-pre-processing-7677b774bb-wg7gq\" (UID: \"ed1d004c-b99f-4c1c-b665-86e51c2d53fb\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:11.646723 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.646704    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed1d004c-b99f-4c1c-b665-86e51c2d53fb-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wg7gq\" (UID: \"ed1d004c-b99f-4c1c-b665-86e51c2d53fb\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:11.654265 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.654246    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9kwl\" (UniqueName: \"kubernetes.io/projected/ed1d004c-b99f-4c1c-b665-86e51c2d53fb-kube-api-access-c9kwl\") pod \"payload-pre-processing-7677b774bb-wg7gq\" (UID: \"ed1d004c-b99f-4c1c-b665-86e51c2d53fb\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:11.715415 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.715383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:11.852946 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:11.852923    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"]
Jul 18 01:58:11.855491 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:58:11.855465    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1d004c_b99f_4c1c_b665_86e51c2d53fb.slice/crio-bdee0785032b67e8aa872f480fec99407a34c8abdcc30574f94f1c7d634b2c4b WatchSource:0}: Error finding container bdee0785032b67e8aa872f480fec99407a34c8abdcc30574f94f1c7d634b2c4b: Status 404 returned error can't find the container with id bdee0785032b67e8aa872f480fec99407a34c8abdcc30574f94f1c7d634b2c4b
Jul 18 01:58:12.116852 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:12.116774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq" event={"ID":"ed1d004c-b99f-4c1c-b665-86e51c2d53fb","Type":"ContainerStarted","Data":"bdee0785032b67e8aa872f480fec99407a34c8abdcc30574f94f1c7d634b2c4b"}
Jul 18 01:58:14.126505 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:14.126468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq" event={"ID":"ed1d004c-b99f-4c1c-b665-86e51c2d53fb","Type":"ContainerStarted","Data":"6bd75374307825750b4629863115c5a9d9ac4e8ba327da4b4e0e310f95605228"}
Jul 18 01:58:14.126979 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:14.126584    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:14.143944 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:14.143897    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq" podStartSLOduration=1.695954951 podStartE2EDuration="3.143881111s" totalImagesPullingTime="1.44792616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:14.142956428 +0000 UTC m=+587.352670640" watchObservedRunningTime="2026-07-18 01:58:14.143881111 +0000 UTC m=+587.353595388"
Jul 18 01:58:20.595490 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:20.595455    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:21.773372 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:21.773340    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:25.133473 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:25.133442    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-wg7gq"
Jul 18 01:58:28.396184 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:28.396147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:38.301974 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.301943    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"]
Jul 18 01:58:38.305401 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.305381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.309413 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.309391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 18 01:58:38.309643 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.309625    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mb98f\""
Jul 18 01:58:38.309740 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.309675    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 18 01:58:38.309740 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.309715    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 18 01:58:38.314115 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.314091    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"]
Jul 18 01:58:38.323020 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.322999    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:38.380893 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.380861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jz95\" (UniqueName: \"kubernetes.io/projected/301922eb-9209-4660-b46d-47361cc5c326-kube-api-access-6jz95\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.381029 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.380905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.381029 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.380970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/301922eb-9209-4660-b46d-47361cc5c326-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.381029 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.380989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.381029 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.381014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.381217 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.381108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.481931 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.481896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/301922eb-9209-4660-b46d-47361cc5c326-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.481931 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.481932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.482222 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.481966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.482222 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.482070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.482222 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.482152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jz95\" (UniqueName: \"kubernetes.io/projected/301922eb-9209-4660-b46d-47361cc5c326-kube-api-access-6jz95\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.482222 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.482202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.482448 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.482419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.482506 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.482454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.482622 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.482593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.484456 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.484430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/301922eb-9209-4660-b46d-47361cc5c326-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.484690 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.484674    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/301922eb-9209-4660-b46d-47361cc5c326-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.490935 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.490913    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jz95\" (UniqueName: \"kubernetes.io/projected/301922eb-9209-4660-b46d-47361cc5c326-kube-api-access-6jz95\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5m4pz\" (UID: \"301922eb-9209-4660-b46d-47361cc5c326\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.616650 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.616547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:38.755828 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:58:38.755800    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301922eb_9209_4660_b46d_47361cc5c326.slice/crio-047bdc06ce0aa4b2fd355d75b6c15a8a324312b80b98aecad290c012ee76e878 WatchSource:0}: Error finding container 047bdc06ce0aa4b2fd355d75b6c15a8a324312b80b98aecad290c012ee76e878: Status 404 returned error can't find the container with id 047bdc06ce0aa4b2fd355d75b6c15a8a324312b80b98aecad290c012ee76e878
Jul 18 01:58:38.761560 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:38.761531    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"]
Jul 18 01:58:39.239356 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:39.239315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz" event={"ID":"301922eb-9209-4660-b46d-47361cc5c326","Type":"ContainerStarted","Data":"047bdc06ce0aa4b2fd355d75b6c15a8a324312b80b98aecad290c012ee76e878"}
Jul 18 01:58:40.244774 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:40.244728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz" event={"ID":"301922eb-9209-4660-b46d-47361cc5c326","Type":"ContainerStarted","Data":"fa0c93179af1eaf8da42379a516fdb092152784484c4baf77daa243d3a5a69c5"}
Jul 18 01:58:40.263010 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:40.262955    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz" podStartSLOduration=1.278913115 podStartE2EDuration="2.262931473s" totalImagesPullingTime="984.018358ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:40.261132497 +0000 UTC m=+613.470846728" watchObservedRunningTime="2026-07-18 01:58:40.262931473 +0000 UTC m=+613.472645714"
Jul 18 01:58:45.028494 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:45.028459    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:48.617746 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:48.617648    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:48.617746 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:48.617743    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:48.629997 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:48.629971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:49.296517 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:49.296490    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5m4pz"
Jul 18 01:58:49.728151 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:49.728115    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:54.541716 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:54.541683    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:59.604636 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.604605    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"]
Jul 18 01:58:59.608342 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.608325    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.611290 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.611266    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 18 01:58:59.615880 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.615857    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"]
Jul 18 01:58:59.629851 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.629817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:59.666405 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.666374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.666405 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.666410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.666572 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.666477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.666572 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.666499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.666572 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.666540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.666572 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.666562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bknnn\" (UniqueName: \"kubernetes.io/projected/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-kube-api-access-bknnn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.767972 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.767935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.767972 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.767973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.768217 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.768017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.768217 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.768055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.768217 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.768082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.768368 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.768223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bknnn\" (UniqueName: \"kubernetes.io/projected/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-kube-api-access-bknnn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.768430 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.768403    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.768493 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.768460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.768551 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.768532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.770493 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.770477    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.770648 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.770633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.776650 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.776622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bknnn\" (UniqueName: \"kubernetes.io/projected/5f3a31b2-f2f8-4004-8ba9-aea20c69c193-kube-api-access-bknnn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc\" (UID: \"5f3a31b2-f2f8-4004-8ba9-aea20c69c193\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:58:59.919165 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:58:59.919131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:59:00.047911 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:00.047844    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"]
Jul 18 01:59:00.050267 ip-10-0-142-56 kubenswrapper[2580]: W0718 01:59:00.050239    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f3a31b2_f2f8_4004_8ba9_aea20c69c193.slice/crio-5bcd8e3846cefcd46c2126eb309059a9dd83dd26752b72360b342a550383bf07 WatchSource:0}: Error finding container 5bcd8e3846cefcd46c2126eb309059a9dd83dd26752b72360b342a550383bf07: Status 404 returned error can't find the container with id 5bcd8e3846cefcd46c2126eb309059a9dd83dd26752b72360b342a550383bf07
Jul 18 01:59:00.051742 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:00.051725    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:59:00.334316 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:00.334284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc" event={"ID":"5f3a31b2-f2f8-4004-8ba9-aea20c69c193","Type":"ContainerStarted","Data":"f23148c8d54851a223843710c533af34eb54cef69dda4579185749d379ceb630"}
Jul 18 01:59:00.334477 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:00.334343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc" event={"ID":"5f3a31b2-f2f8-4004-8ba9-aea20c69c193","Type":"ContainerStarted","Data":"5bcd8e3846cefcd46c2126eb309059a9dd83dd26752b72360b342a550383bf07"}
Jul 18 01:59:00.352287 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:00.352240    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc" podStartSLOduration=1.18187953 podStartE2EDuration="1.35221946s" totalImagesPullingTime="170.33993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:59:00.350191439 +0000 UTC m=+633.559905657" watchObservedRunningTime="2026-07-18 01:59:00.35221946 +0000 UTC m=+633.561933679"
Jul 18 01:59:09.920060 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:09.920010    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:59:09.920526 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:09.920127    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:59:09.932426 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:09.932400    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:59:10.126481 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:10.126448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:59:10.397003 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:10.396929    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc"
Jul 18 01:59:16.612300 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:16.612267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 01:59:52.864312 ip-10-0-142-56 kubenswrapper[2580]: I0718 01:59:52.864277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:24.505085 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:00:24.504978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:24.605329 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:00:24.605301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:25.697169 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:00:25.697140    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:25.797119 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:00:25.797086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:37.536983 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:00:37.536952    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:37.593377 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:00:37.593341    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:41.105777 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:00:41.105720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:01:18.091596 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:01:18.091528    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:01:18.193227 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:01:18.193175    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:30.294274 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:02:30.294238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:31.790307 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:02:31.790269    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:51.528635 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:02:51.528607    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:11.886646 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:11.886616    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:11.985088 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:11.985064    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:12.115682 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:12.115644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:17.933211 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:17.933121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:18.041960 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:18.041929    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:18.131539 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:18.131506    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:18.263735 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:18.263646    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:23.889468 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:23.889431    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:25.561370 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:25.561332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:58.159866 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:03:58.159835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:04:50.744241 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:04:50.744162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:06:08.422004 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:06:08.421964    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:06:08.474066 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:06:08.474014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:11:48.567845 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:11:48.567814    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:11:49.765265 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:11:49.765218    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:11:49.978451 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:11:49.978418    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:24.372443 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:24.372354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:33.182220 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:33.182185    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:41.435827 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:41.435788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:46.511532 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:46.511496    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:49.413264 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:49.413217    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:49.502620 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:49.502577    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:49.844813 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:49.844778    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:55.632199 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:55.632160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:55.773474 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:55.773446    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:55.872628 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:55.872591    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:56.159154 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:56.159116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:56.384221 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:56.384187    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:58.906324 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:58.906288    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:58.981893 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:58.981850    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:58.983100 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:58.983077    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"]
Jul 18 02:14:58.986364 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:58.986344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:58.989448 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:58.989433    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c62b43cb-serving-cert\""
Jul 18 02:14:58.990147 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:58.990126    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:14:58.990250 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:58.990182    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:14:58.990250 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:58.990186    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:14:58.996164 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:58.996143    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"]
Jul 18 02:14:59.056464 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.056434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:59.114864 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.114831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdht6\" (UniqueName: \"kubernetes.io/projected/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-kube-api-access-pdht6\") pod \"maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.115027 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.114879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-openshift-service-ca\") pod \"maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.115027 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.114955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-maas-api-tls\") pod \"maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.212812 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.212781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:59.215649 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.215630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-maas-api-tls\") pod \"maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.215728 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.215694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdht6\" (UniqueName: \"kubernetes.io/projected/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-kube-api-access-pdht6\") pod \"maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.215772 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.215730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-openshift-service-ca\") pod \"maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.216491 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.216391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-openshift-service-ca\") pod \"maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.218204 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.218185    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-maas-api-tls\") pod \"maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.225093 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.225073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdht6\" (UniqueName: \"kubernetes.io/projected/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-kube-api-access-pdht6\") pod \"maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.297114 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.297079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:14:59.433439 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.433415    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"]
Jul 18 02:14:59.436186 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:14:59.436154    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f1cd13_353b_4d54_bedd_1abd5dd73c89.slice/crio-3217f3d1d42a8c056888f9b1ba6d0174b56dfb4abe014b3bfae1e0dff6c09a44 WatchSource:0}: Error finding container 3217f3d1d42a8c056888f9b1ba6d0174b56dfb4abe014b3bfae1e0dff6c09a44: Status 404 returned error can't find the container with id 3217f3d1d42a8c056888f9b1ba6d0174b56dfb4abe014b3bfae1e0dff6c09a44
Jul 18 02:14:59.437829 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.437812    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:14:59.552252 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.552168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:59.655494 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.655391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:59.729865 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:14:59.729823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p" event={"ID":"c1f1cd13-353b-4d54-bedd-1abd5dd73c89","Type":"ContainerStarted","Data":"3217f3d1d42a8c056888f9b1ba6d0174b56dfb4abe014b3bfae1e0dff6c09a44"}
Jul 18 02:15:00.136183 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:00.136153    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"]
Jul 18 02:15:00.139641 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:00.139624    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"
Jul 18 02:15:00.143258 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:00.143234    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:15:00.151318 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:00.151294    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"]
Jul 18 02:15:00.151444 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:00.151305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"
Jul 18 02:15:00.303883 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:00.303675    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"]
Jul 18 02:15:00.306394 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:15:00.306366    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda033eb84_f61f_42e7_8785_4b214b44ddcc.slice/crio-03285a0ad1f506b0adf5aa1774caa955041338460594aad07d6a8055d2b2ad30 WatchSource:0}: Error finding container 03285a0ad1f506b0adf5aa1774caa955041338460594aad07d6a8055d2b2ad30: Status 404 returned error can't find the container with id 03285a0ad1f506b0adf5aa1774caa955041338460594aad07d6a8055d2b2ad30
Jul 18 02:15:00.735488 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:00.735451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" event={"ID":"a033eb84-f61f-42e7-8785-4b214b44ddcc","Type":"ContainerStarted","Data":"03285a0ad1f506b0adf5aa1774caa955041338460594aad07d6a8055d2b2ad30"}
Jul 18 02:15:01.743413 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:01.743378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p" event={"ID":"c1f1cd13-353b-4d54-bedd-1abd5dd73c89","Type":"ContainerStarted","Data":"891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253"}
Jul 18 02:15:01.743880 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:01.743501    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:15:01.761519 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:01.761464    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p" podStartSLOduration=1.8798001869999998 podStartE2EDuration="3.761446443s" totalImagesPullingTime="1.881646256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:01.759644263 +0000 UTC m=+1594.969358483" watchObservedRunningTime="2026-07-18 02:15:01.761446443 +0000 UTC m=+1594.971160670"
Jul 18 02:15:02.750010 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:02.749975    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" event={"ID":"a033eb84-f61f-42e7-8785-4b214b44ddcc","Type":"ContainerStarted","Data":"cf324ab86a51e42635e3385d81a5362b01d29f9876fb0c65d6ddded3e383ef2a"}
Jul 18 02:15:02.765271 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:02.765223    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" podStartSLOduration=0.745439288 podStartE2EDuration="2.765206802s" totalImagesPullingTime="2.019767514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:02.764119587 +0000 UTC m=+1595.973833807" watchObservedRunningTime="2026-07-18 02:15:02.765206802 +0000 UTC m=+1595.974921022"
Jul 18 02:15:03.756306 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:03.756267    2580 generic.go:356] "Generic (PLEG): container finished" podID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerID="cf324ab86a51e42635e3385d81a5362b01d29f9876fb0c65d6ddded3e383ef2a" exitCode=7
Jul 18 02:15:03.756657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:03.756353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" event={"ID":"a033eb84-f61f-42e7-8785-4b214b44ddcc","Type":"ContainerDied","Data":"cf324ab86a51e42635e3385d81a5362b01d29f9876fb0c65d6ddded3e383ef2a"}
Jul 18 02:15:03.756657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:03.756469    2580 scope.go:117] "RemoveContainer" containerID="cf324ab86a51e42635e3385d81a5362b01d29f9876fb0c65d6ddded3e383ef2a"
Jul 18 02:15:04.762337 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:04.762304    2580 generic.go:356] "Generic (PLEG): container finished" podID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerID="03f7b22b6fb8deff042a6d8f855fd8b763c5bf19153fa214532dc910320bb9a5" exitCode=7
Jul 18 02:15:04.762772 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:04.762374    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" event={"ID":"a033eb84-f61f-42e7-8785-4b214b44ddcc","Type":"ContainerDied","Data":"03f7b22b6fb8deff042a6d8f855fd8b763c5bf19153fa214532dc910320bb9a5"}
Jul 18 02:15:04.762772 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:04.762436    2580 scope.go:117] "RemoveContainer" containerID="cf324ab86a51e42635e3385d81a5362b01d29f9876fb0c65d6ddded3e383ef2a"
Jul 18 02:15:04.762772 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:04.762567    2580 scope.go:117] "RemoveContainer" containerID="03f7b22b6fb8deff042a6d8f855fd8b763c5bf19153fa214532dc910320bb9a5"
Jul 18 02:15:04.762928 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:15:04.762820    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w_odh-ai-gateway-infra(a033eb84-f61f-42e7-8785-4b214b44ddcc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc"
Jul 18 02:15:05.768850 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:05.768821    2580 scope.go:117] "RemoveContainer" containerID="03f7b22b6fb8deff042a6d8f855fd8b763c5bf19153fa214532dc910320bb9a5"
Jul 18 02:15:05.769248 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:15:05.769034    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w_odh-ai-gateway-infra(a033eb84-f61f-42e7-8785-4b214b44ddcc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc"
Jul 18 02:15:07.755189 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:07.755163    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:15:17.388372 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:17.388297    2580 scope.go:117] "RemoveContainer" containerID="03f7b22b6fb8deff042a6d8f855fd8b763c5bf19153fa214532dc910320bb9a5"
Jul 18 02:15:17.832435 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:17.832404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" event={"ID":"a033eb84-f61f-42e7-8785-4b214b44ddcc","Type":"ContainerStarted","Data":"b01e6e457ef4e74dd1a6ebf73c52025a56a8f3026f4db6cb12bc7245a8094946"}
Jul 18 02:15:18.839129 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:18.839094    2580 generic.go:356] "Generic (PLEG): container finished" podID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerID="b01e6e457ef4e74dd1a6ebf73c52025a56a8f3026f4db6cb12bc7245a8094946" exitCode=0
Jul 18 02:15:18.839551 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:18.839167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" event={"ID":"a033eb84-f61f-42e7-8785-4b214b44ddcc","Type":"ContainerDied","Data":"b01e6e457ef4e74dd1a6ebf73c52025a56a8f3026f4db6cb12bc7245a8094946"}
Jul 18 02:15:18.839551 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:18.839226    2580 scope.go:117] "RemoveContainer" containerID="03f7b22b6fb8deff042a6d8f855fd8b763c5bf19153fa214532dc910320bb9a5"
Jul 18 02:15:18.864186 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:18.864157    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"]
Jul 18 02:15:19.986363 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:19.986341    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"
Jul 18 02:15:20.850333 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:20.850300    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w" event={"ID":"a033eb84-f61f-42e7-8785-4b214b44ddcc","Type":"ContainerDied","Data":"03285a0ad1f506b0adf5aa1774caa955041338460594aad07d6a8055d2b2ad30"}
Jul 18 02:15:20.850333 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:20.850320    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"
Jul 18 02:15:20.850549 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:20.850357    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03285a0ad1f506b0adf5aa1774caa955041338460594aad07d6a8055d2b2ad30"
Jul 18 02:15:20.878974 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:20.878944    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"]
Jul 18 02:15:20.884387 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:20.884363    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c62b43cb-29739015-k5b5w"]
Jul 18 02:15:21.390957 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:21.390924    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" path="/var/lib/kubelet/pods/a033eb84-f61f-42e7-8785-4b214b44ddcc/volumes"
Jul 18 02:15:34.834173 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:34.834139    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"]
Jul 18 02:15:34.834655 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:34.834353    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p" podUID="c1f1cd13-353b-4d54-bedd-1abd5dd73c89" containerName="maas-api" containerID="cri-o://891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253" gracePeriod=30
Jul 18 02:15:34.860485 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:34.860452    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:35.076495 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.076468    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:15:35.137999 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.137913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdht6\" (UniqueName: \"kubernetes.io/projected/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-kube-api-access-pdht6\") pod \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") "
Jul 18 02:15:35.137999 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.137967    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-openshift-service-ca\") pod \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") "
Jul 18 02:15:35.138272 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.138011    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-maas-api-tls\") pod \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\" (UID: \"c1f1cd13-353b-4d54-bedd-1abd5dd73c89\") "
Jul 18 02:15:35.138334 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.138313    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1f1cd13-353b-4d54-bedd-1abd5dd73c89" (UID: "c1f1cd13-353b-4d54-bedd-1abd5dd73c89"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:15:35.140341 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.140312    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-kube-api-access-pdht6" (OuterVolumeSpecName: "kube-api-access-pdht6") pod "c1f1cd13-353b-4d54-bedd-1abd5dd73c89" (UID: "c1f1cd13-353b-4d54-bedd-1abd5dd73c89"). InnerVolumeSpecName "kube-api-access-pdht6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:15:35.140453 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.140391    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1f1cd13-353b-4d54-bedd-1abd5dd73c89" (UID: "c1f1cd13-353b-4d54-bedd-1abd5dd73c89"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:15:35.238829 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.238792    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdht6\" (UniqueName: \"kubernetes.io/projected/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-kube-api-access-pdht6\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:15:35.238829 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.238823    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:15:35.238829 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.238833    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1f1cd13-353b-4d54-bedd-1abd5dd73c89-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:15:35.917338 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.917304    2580 generic.go:356] "Generic (PLEG): container finished" podID="c1f1cd13-353b-4d54-bedd-1abd5dd73c89" containerID="891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253" exitCode=0
Jul 18 02:15:35.917838 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.917360    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"
Jul 18 02:15:35.917838 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.917386    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p" event={"ID":"c1f1cd13-353b-4d54-bedd-1abd5dd73c89","Type":"ContainerDied","Data":"891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253"}
Jul 18 02:15:35.917838 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.917431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p" event={"ID":"c1f1cd13-353b-4d54-bedd-1abd5dd73c89","Type":"ContainerDied","Data":"3217f3d1d42a8c056888f9b1ba6d0174b56dfb4abe014b3bfae1e0dff6c09a44"}
Jul 18 02:15:35.917838 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.917454    2580 scope.go:117] "RemoveContainer" containerID="891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253"
Jul 18 02:15:35.926131 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.926112    2580 scope.go:117] "RemoveContainer" containerID="891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253"
Jul 18 02:15:35.926392 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:15:35.926373    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253\": container with ID starting with 891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253 not found: ID does not exist" containerID="891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253"
Jul 18 02:15:35.926470 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.926400    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253"} err="failed to get container status \"891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253\": rpc error: code = NotFound desc = could not find container \"891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253\": container with ID starting with 891b62920af168afee284b20888910e17d0bd5f4e9ec5fe5251e5d9afafe3253 not found: ID does not exist"
Jul 18 02:15:35.933821 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.933790    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"]
Jul 18 02:15:35.939240 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:35.939219    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c62b43cb-7d65d8c6b8-bqr7p"]
Jul 18 02:15:37.390737 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:37.390702    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f1cd13-353b-4d54-bedd-1abd5dd73c89" path="/var/lib/kubelet/pods/c1f1cd13-353b-4d54-bedd-1abd5dd73c89/volumes"
Jul 18 02:15:39.512814 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:39.512779    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:39.659109 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:39.659076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:39.888171 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:39.888077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:40.139892 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:40.139813    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:40.236215 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:40.236173    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:40.351451 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:40.351408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:40.669718 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:40.669682    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:40.993136 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:40.993032    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:43.362459 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.362423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:43.442008 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.441972    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"]
Jul 18 02:15:43.442484 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442467    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f1cd13-353b-4d54-bedd-1abd5dd73c89" containerName="maas-api"
Jul 18 02:15:43.442484 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442486    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f1cd13-353b-4d54-bedd-1abd5dd73c89" containerName="maas-api"
Jul 18 02:15:43.442597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442504    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerName="cleanup"
Jul 18 02:15:43.442597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442510    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerName="cleanup"
Jul 18 02:15:43.442597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442528    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerName="cleanup"
Jul 18 02:15:43.442597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442534    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerName="cleanup"
Jul 18 02:15:43.442597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442543    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerName="cleanup"
Jul 18 02:15:43.442597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442548    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerName="cleanup"
Jul 18 02:15:43.442780 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442607    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f1cd13-353b-4d54-bedd-1abd5dd73c89" containerName="maas-api"
Jul 18 02:15:43.442780 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442615    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerName="cleanup"
Jul 18 02:15:43.442780 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442622    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerName="cleanup"
Jul 18 02:15:43.442780 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.442627    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a033eb84-f61f-42e7-8785-4b214b44ddcc" containerName="cleanup"
Jul 18 02:15:43.445748 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.445718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.448482 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.448464    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:15:43.448597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.448464    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:15:43.449478 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.449463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:15:43.449545 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.449473    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-23b07985-serving-cert\""
Jul 18 02:15:43.454408 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.454379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"]
Jul 18 02:15:43.465108 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.465074    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:43.513985 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.513949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-maas-api-tls\") pod \"maas-api-e2e-migrate-23b07985-64b4f95b66-j4672\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.514191 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.514160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-openshift-service-ca\") pod \"maas-api-e2e-migrate-23b07985-64b4f95b66-j4672\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.514247 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.514194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ts2x\" (UniqueName: \"kubernetes.io/projected/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-kube-api-access-4ts2x\") pod \"maas-api-e2e-migrate-23b07985-64b4f95b66-j4672\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.615727 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.615629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-openshift-service-ca\") pod \"maas-api-e2e-migrate-23b07985-64b4f95b66-j4672\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.615727 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.615668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ts2x\" (UniqueName: \"kubernetes.io/projected/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-kube-api-access-4ts2x\") pod \"maas-api-e2e-migrate-23b07985-64b4f95b66-j4672\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.615976 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.615800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-maas-api-tls\") pod \"maas-api-e2e-migrate-23b07985-64b4f95b66-j4672\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.616350 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.616328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-openshift-service-ca\") pod \"maas-api-e2e-migrate-23b07985-64b4f95b66-j4672\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.618441 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.618419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-maas-api-tls\") pod \"maas-api-e2e-migrate-23b07985-64b4f95b66-j4672\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.623486 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.623455    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:43.624320 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.624296    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ts2x\" (UniqueName: \"kubernetes.io/projected/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-kube-api-access-4ts2x\") pod \"maas-api-e2e-migrate-23b07985-64b4f95b66-j4672\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.756886 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.756838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:43.898449 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.898421    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"]
Jul 18 02:15:43.900978 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:15:43.900947    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51162d1e_fd7c_4d9a_b32a_eacf35c91ac1.slice/crio-20b54da90713db04e6e6466b14e8565f90da2df85454553632f4f8902bd3a587 WatchSource:0}: Error finding container 20b54da90713db04e6e6466b14e8565f90da2df85454553632f4f8902bd3a587: Status 404 returned error can't find the container with id 20b54da90713db04e6e6466b14e8565f90da2df85454553632f4f8902bd3a587
Jul 18 02:15:43.957248 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.957211    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672" event={"ID":"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1","Type":"ContainerStarted","Data":"20b54da90713db04e6e6466b14e8565f90da2df85454553632f4f8902bd3a587"}
Jul 18 02:15:43.958094 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:43.958022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:44.058848 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:44.058804    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:44.964449 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:44.964405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672" event={"ID":"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1","Type":"ContainerStarted","Data":"7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d"}
Jul 18 02:15:44.964824 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:44.964484    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:15:44.981819 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:44.981769    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672" podStartSLOduration=1.596801063 podStartE2EDuration="1.98175568s" totalImagesPullingTime="384.954617ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:44.980930395 +0000 UTC m=+1638.190644615" watchObservedRunningTime="2026-07-18 02:15:44.98175568 +0000 UTC m=+1638.191469935"
Jul 18 02:15:50.974461 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:15:50.974430    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:16:08.377158 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:08.377110    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"]
Jul 18 02:16:08.380729 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:08.380711    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"
Jul 18 02:16:08.383463 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:08.383442    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:16:08.387189 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:08.387169    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"]
Jul 18 02:16:08.429886 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:08.429857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5863909b-0654-4c1f-b106-87c9cdbb7e60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-23b07985-656mk\" (UID: \"5863909b-0654-4c1f-b106-87c9cdbb7e60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"
Jul 18 02:16:08.530945 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:08.530917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5863909b-0654-4c1f-b106-87c9cdbb7e60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-23b07985-656mk\" (UID: \"5863909b-0654-4c1f-b106-87c9cdbb7e60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"
Jul 18 02:16:08.531547 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:08.531528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5863909b-0654-4c1f-b106-87c9cdbb7e60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-23b07985-656mk\" (UID: \"5863909b-0654-4c1f-b106-87c9cdbb7e60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"
Jul 18 02:16:08.693598 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:08.693553    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"
Jul 18 02:16:08.824444 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:08.824288    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"]
Jul 18 02:16:08.827144 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:16:08.827103    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5863909b_0654_4c1f_b106_87c9cdbb7e60.slice/crio-5437f21b3d786788521c27bbd30a1bfd335da07612740977df65ec27e477318d WatchSource:0}: Error finding container 5437f21b3d786788521c27bbd30a1bfd335da07612740977df65ec27e477318d: Status 404 returned error can't find the container with id 5437f21b3d786788521c27bbd30a1bfd335da07612740977df65ec27e477318d
Jul 18 02:16:09.080031 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:09.079995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk" event={"ID":"5863909b-0654-4c1f-b106-87c9cdbb7e60","Type":"ContainerStarted","Data":"5437f21b3d786788521c27bbd30a1bfd335da07612740977df65ec27e477318d"}
Jul 18 02:16:10.086185 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:10.086146    2580 generic.go:356] "Generic (PLEG): container finished" podID="5863909b-0654-4c1f-b106-87c9cdbb7e60" containerID="828935eae3b247edf4f8ca873c0cb04aca7fdae0ac6a96a2eb200de62bb1380b" exitCode=0
Jul 18 02:16:10.086570 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:10.086226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk" event={"ID":"5863909b-0654-4c1f-b106-87c9cdbb7e60","Type":"ContainerDied","Data":"828935eae3b247edf4f8ca873c0cb04aca7fdae0ac6a96a2eb200de62bb1380b"}
Jul 18 02:16:11.222728 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:11.222705    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"
Jul 18 02:16:11.254955 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:11.254923    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5863909b-0654-4c1f-b106-87c9cdbb7e60-maas-api-service-ca\") pod \"5863909b-0654-4c1f-b106-87c9cdbb7e60\" (UID: \"5863909b-0654-4c1f-b106-87c9cdbb7e60\") "
Jul 18 02:16:11.255354 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:11.255331    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5863909b-0654-4c1f-b106-87c9cdbb7e60-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5863909b-0654-4c1f-b106-87c9cdbb7e60" (UID: "5863909b-0654-4c1f-b106-87c9cdbb7e60"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:16:11.356006 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:11.355918    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5863909b-0654-4c1f-b106-87c9cdbb7e60-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:16:12.097586 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:12.097557    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"
Jul 18 02:16:12.097586 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:12.097571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk" event={"ID":"5863909b-0654-4c1f-b106-87c9cdbb7e60","Type":"ContainerDied","Data":"5437f21b3d786788521c27bbd30a1bfd335da07612740977df65ec27e477318d"}
Jul 18 02:16:12.097815 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:12.097649    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5437f21b3d786788521c27bbd30a1bfd335da07612740977df65ec27e477318d"
Jul 18 02:16:18.758340 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:18.758301    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"]
Jul 18 02:16:18.759124 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:18.758564    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672" podUID="51162d1e-fd7c-4d9a-b32a-eacf35c91ac1" containerName="maas-api" containerID="cri-o://7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d" gracePeriod=30
Jul 18 02:16:18.783316 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:18.783290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:19.009127 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.009069    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:16:19.122471 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.122435    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-openshift-service-ca\") pod \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") "
Jul 18 02:16:19.122667 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.122528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ts2x\" (UniqueName: \"kubernetes.io/projected/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-kube-api-access-4ts2x\") pod \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") "
Jul 18 02:16:19.122667 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.122560    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-maas-api-tls\") pod \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\" (UID: \"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1\") "
Jul 18 02:16:19.122825 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.122799    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "51162d1e-fd7c-4d9a-b32a-eacf35c91ac1" (UID: "51162d1e-fd7c-4d9a-b32a-eacf35c91ac1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:16:19.124674 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.124643    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "51162d1e-fd7c-4d9a-b32a-eacf35c91ac1" (UID: "51162d1e-fd7c-4d9a-b32a-eacf35c91ac1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:16:19.124788 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.124730    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-kube-api-access-4ts2x" (OuterVolumeSpecName: "kube-api-access-4ts2x") pod "51162d1e-fd7c-4d9a-b32a-eacf35c91ac1" (UID: "51162d1e-fd7c-4d9a-b32a-eacf35c91ac1"). InnerVolumeSpecName "kube-api-access-4ts2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:16:19.139245 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.139218    2580 generic.go:356] "Generic (PLEG): container finished" podID="51162d1e-fd7c-4d9a-b32a-eacf35c91ac1" containerID="7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d" exitCode=0
Jul 18 02:16:19.139355 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.139262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672" event={"ID":"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1","Type":"ContainerDied","Data":"7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d"}
Jul 18 02:16:19.139355 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.139326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672" event={"ID":"51162d1e-fd7c-4d9a-b32a-eacf35c91ac1","Type":"ContainerDied","Data":"20b54da90713db04e6e6466b14e8565f90da2df85454553632f4f8902bd3a587"}
Jul 18 02:16:19.139355 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.139331    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"
Jul 18 02:16:19.139355 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.139347    2580 scope.go:117] "RemoveContainer" containerID="7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d"
Jul 18 02:16:19.148087 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.148070    2580 scope.go:117] "RemoveContainer" containerID="7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d"
Jul 18 02:16:19.148331 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:16:19.148313    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d\": container with ID starting with 7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d not found: ID does not exist" containerID="7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d"
Jul 18 02:16:19.148369 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.148339    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d"} err="failed to get container status \"7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d\": rpc error: code = NotFound desc = could not find container \"7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d\": container with ID starting with 7366c714e47367bbebb89c3692a50e7a4dba58222c4a7c21adb4cb50cf7b014d not found: ID does not exist"
Jul 18 02:16:19.161070 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.161029    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"]
Jul 18 02:16:19.164946 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.164927    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23b07985-64b4f95b66-j4672"]
Jul 18 02:16:19.224167 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.224129    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ts2x\" (UniqueName: \"kubernetes.io/projected/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-kube-api-access-4ts2x\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:16:19.224167 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.224160    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:16:19.224167 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.224171    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:16:19.390627 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:19.390543    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51162d1e-fd7c-4d9a-b32a-eacf35c91ac1" path="/var/lib/kubelet/pods/51162d1e-fd7c-4d9a-b32a-eacf35c91ac1/volumes"
Jul 18 02:16:23.442115 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:23.442079    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"]
Jul 18 02:16:23.444299 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:23.444282    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-23b07985-656mk"]
Jul 18 02:16:23.447705 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:23.447680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:23.604654 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:23.604616    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:23.849789 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:23.849699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:24.129104 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:24.128998    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:24.397698 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:24.397605    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:24.664857 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:24.664758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:25.390774 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:25.390739    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5863909b-0654-4c1f-b106-87c9cdbb7e60" path="/var/lib/kubelet/pods/5863909b-0654-4c1f-b106-87c9cdbb7e60/volumes"
Jul 18 02:16:26.455178 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.455139    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:26.532307 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.532276    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"]
Jul 18 02:16:26.532970 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.532947    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5863909b-0654-4c1f-b106-87c9cdbb7e60" containerName="revoke-keys"
Jul 18 02:16:26.532970 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.532969    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5863909b-0654-4c1f-b106-87c9cdbb7e60" containerName="revoke-keys"
Jul 18 02:16:26.533193 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.532984    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51162d1e-fd7c-4d9a-b32a-eacf35c91ac1" containerName="maas-api"
Jul 18 02:16:26.533193 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.532990    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="51162d1e-fd7c-4d9a-b32a-eacf35c91ac1" containerName="maas-api"
Jul 18 02:16:26.533193 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.533061    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5863909b-0654-4c1f-b106-87c9cdbb7e60" containerName="revoke-keys"
Jul 18 02:16:26.533193 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.533069    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="51162d1e-fd7c-4d9a-b32a-eacf35c91ac1" containerName="maas-api"
Jul 18 02:16:26.534642 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.534615    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:26.536208 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.536191    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:26.538983 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.538962    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9a88968c-serving-cert\""
Jul 18 02:16:26.540265 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.540231    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:16:26.540374 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.540266    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:16:26.540374 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.540344    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:16:26.547150 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.547129    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"]
Jul 18 02:16:26.591030 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.590997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f81dc3d-43c4-4899-b605-4579c70a1dca-openshift-service-ca\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:26.591192 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.591086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f81dc3d-43c4-4899-b605-4579c70a1dca-maas-api-tls\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:26.591192 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.591116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hvwz\" (UniqueName: \"kubernetes.io/projected/8f81dc3d-43c4-4899-b605-4579c70a1dca-kube-api-access-9hvwz\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:26.599657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.599631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:26.692427 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.692392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f81dc3d-43c4-4899-b605-4579c70a1dca-maas-api-tls\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:26.692564 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.692446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hvwz\" (UniqueName: \"kubernetes.io/projected/8f81dc3d-43c4-4899-b605-4579c70a1dca-kube-api-access-9hvwz\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:26.692564 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.692540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f81dc3d-43c4-4899-b605-4579c70a1dca-openshift-service-ca\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:26.692639 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:16:26.692542    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-serving-cert: secret "maas-api-e2e-ait-9a88968c-serving-cert" not found
Jul 18 02:16:26.692677 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:16:26.692655    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f81dc3d-43c4-4899-b605-4579c70a1dca-maas-api-tls podName:8f81dc3d-43c4-4899-b605-4579c70a1dca nodeName:}" failed. No retries permitted until 2026-07-18 02:16:27.192637742 +0000 UTC m=+1680.402351944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8f81dc3d-43c4-4899-b605-4579c70a1dca-maas-api-tls") pod "maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv" (UID: "8f81dc3d-43c4-4899-b605-4579c70a1dca") : secret "maas-api-e2e-ait-9a88968c-serving-cert" not found
Jul 18 02:16:26.693336 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.693313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f81dc3d-43c4-4899-b605-4579c70a1dca-openshift-service-ca\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:26.702844 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.702823    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hvwz\" (UniqueName: \"kubernetes.io/projected/8f81dc3d-43c4-4899-b605-4579c70a1dca-kube-api-access-9hvwz\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:26.994804 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:26.994759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:27.099691 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:27.099651    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:27.160239 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:27.160203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:27.196584 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:27.196556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f81dc3d-43c4-4899-b605-4579c70a1dca-maas-api-tls\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:27.199053 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:27.199028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f81dc3d-43c4-4899-b605-4579c70a1dca-maas-api-tls\") pod \"maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:27.449663 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:27.449643    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:16:27.457657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:27.457639    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:27.590763 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:27.590740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"]
Jul 18 02:16:27.593097 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:16:27.593035    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f81dc3d_43c4_4899_b605_4579c70a1dca.slice/crio-bd99cfa0b8f8b479fbc1d9b76d3a8f8e2cc085e0538ce596ed8f71fea3adc807 WatchSource:0}: Error finding container bd99cfa0b8f8b479fbc1d9b76d3a8f8e2cc085e0538ce596ed8f71fea3adc807: Status 404 returned error can't find the container with id bd99cfa0b8f8b479fbc1d9b76d3a8f8e2cc085e0538ce596ed8f71fea3adc807
Jul 18 02:16:28.185974 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:28.185939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv" event={"ID":"8f81dc3d-43c4-4899-b605-4579c70a1dca","Type":"ContainerStarted","Data":"ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1"}
Jul 18 02:16:28.186166 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:28.185997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv" event={"ID":"8f81dc3d-43c4-4899-b605-4579c70a1dca","Type":"ContainerStarted","Data":"bd99cfa0b8f8b479fbc1d9b76d3a8f8e2cc085e0538ce596ed8f71fea3adc807"}
Jul 18 02:16:28.186166 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:28.186031    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:28.203950 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:28.203905    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv" podStartSLOduration=1.753910369 podStartE2EDuration="2.203892258s" totalImagesPullingTime="449.981889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:16:28.20236503 +0000 UTC m=+1681.412079251" watchObservedRunningTime="2026-07-18 02:16:28.203892258 +0000 UTC m=+1681.413606477"
Jul 18 02:16:34.196836 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:34.196808    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:16:50.634510 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:50.634428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:52.190439 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:52.190399    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"]
Jul 18 02:16:52.194114 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:52.194089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"
Jul 18 02:16:52.196573 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:52.196551    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:16:52.201077 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:52.201019    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"]
Jul 18 02:16:52.222390 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:52.222358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c46def0b-aa3d-414b-bc34-2c7da272a537-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9a88968c-49l86\" (UID: \"c46def0b-aa3d-414b-bc34-2c7da272a537\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"
Jul 18 02:16:52.323741 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:52.323708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c46def0b-aa3d-414b-bc34-2c7da272a537-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9a88968c-49l86\" (UID: \"c46def0b-aa3d-414b-bc34-2c7da272a537\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"
Jul 18 02:16:52.324309 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:52.324292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c46def0b-aa3d-414b-bc34-2c7da272a537-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9a88968c-49l86\" (UID: \"c46def0b-aa3d-414b-bc34-2c7da272a537\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"
Jul 18 02:16:52.505813 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:52.505737    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"
Jul 18 02:16:52.843123 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:52.843102    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"]
Jul 18 02:16:52.845430 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:16:52.845403    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46def0b_aa3d_414b_bc34_2c7da272a537.slice/crio-a3cfced557f0c552320f4fbd4fd9fd6c0a1d4aafaf97630a9dcb5fcfb7da04cd WatchSource:0}: Error finding container a3cfced557f0c552320f4fbd4fd9fd6c0a1d4aafaf97630a9dcb5fcfb7da04cd: Status 404 returned error can't find the container with id a3cfced557f0c552320f4fbd4fd9fd6c0a1d4aafaf97630a9dcb5fcfb7da04cd
Jul 18 02:16:53.300487 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:53.300452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86" event={"ID":"c46def0b-aa3d-414b-bc34-2c7da272a537","Type":"ContainerStarted","Data":"218244cfaf0dfe6d2f628f6660d6a02ba87463643b03d1894ad369bf1f2930ec"}
Jul 18 02:16:53.300917 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:53.300503    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86" event={"ID":"c46def0b-aa3d-414b-bc34-2c7da272a537","Type":"ContainerStarted","Data":"a3cfced557f0c552320f4fbd4fd9fd6c0a1d4aafaf97630a9dcb5fcfb7da04cd"}
Jul 18 02:16:53.317346 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:53.317300    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86" podStartSLOduration=1.3172848 podStartE2EDuration="1.3172848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:16:53.316128292 +0000 UTC m=+1706.525842513" watchObservedRunningTime="2026-07-18 02:16:53.3172848 +0000 UTC m=+1706.526999021"
Jul 18 02:16:54.306664 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:54.306630    2580 generic.go:356] "Generic (PLEG): container finished" podID="c46def0b-aa3d-414b-bc34-2c7da272a537" containerID="218244cfaf0dfe6d2f628f6660d6a02ba87463643b03d1894ad369bf1f2930ec" exitCode=0
Jul 18 02:16:54.307060 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:54.306676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86" event={"ID":"c46def0b-aa3d-414b-bc34-2c7da272a537","Type":"ContainerDied","Data":"218244cfaf0dfe6d2f628f6660d6a02ba87463643b03d1894ad369bf1f2930ec"}
Jul 18 02:16:55.441312 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:55.441287    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"
Jul 18 02:16:55.457021 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:55.456976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c46def0b-aa3d-414b-bc34-2c7da272a537-maas-api-service-ca\") pod \"c46def0b-aa3d-414b-bc34-2c7da272a537\" (UID: \"c46def0b-aa3d-414b-bc34-2c7da272a537\") "
Jul 18 02:16:55.457611 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:55.457582    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c46def0b-aa3d-414b-bc34-2c7da272a537-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c46def0b-aa3d-414b-bc34-2c7da272a537" (UID: "c46def0b-aa3d-414b-bc34-2c7da272a537"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:16:55.558441 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:55.558415    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c46def0b-aa3d-414b-bc34-2c7da272a537-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:16:56.316800 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:56.316771    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"
Jul 18 02:16:56.316970 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:56.316774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86" event={"ID":"c46def0b-aa3d-414b-bc34-2c7da272a537","Type":"ContainerDied","Data":"a3cfced557f0c552320f4fbd4fd9fd6c0a1d4aafaf97630a9dcb5fcfb7da04cd"}
Jul 18 02:16:56.316970 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:16:56.316896    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3cfced557f0c552320f4fbd4fd9fd6c0a1d4aafaf97630a9dcb5fcfb7da04cd"
Jul 18 02:17:02.555264 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.555233    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"]
Jul 18 02:17:02.555818 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.555442    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv" podUID="8f81dc3d-43c4-4899-b605-4579c70a1dca" containerName="maas-api" containerID="cri-o://ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1" gracePeriod=30
Jul 18 02:17:02.576232 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.576200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:02.803651 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.803629    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:17:02.923979 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.923953    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hvwz\" (UniqueName: \"kubernetes.io/projected/8f81dc3d-43c4-4899-b605-4579c70a1dca-kube-api-access-9hvwz\") pod \"8f81dc3d-43c4-4899-b605-4579c70a1dca\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") "
Jul 18 02:17:02.924150 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.923993    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f81dc3d-43c4-4899-b605-4579c70a1dca-openshift-service-ca\") pod \"8f81dc3d-43c4-4899-b605-4579c70a1dca\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") "
Jul 18 02:17:02.924150 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.924030    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f81dc3d-43c4-4899-b605-4579c70a1dca-maas-api-tls\") pod \"8f81dc3d-43c4-4899-b605-4579c70a1dca\" (UID: \"8f81dc3d-43c4-4899-b605-4579c70a1dca\") "
Jul 18 02:17:02.924426 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.924402    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f81dc3d-43c4-4899-b605-4579c70a1dca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f81dc3d-43c4-4899-b605-4579c70a1dca" (UID: "8f81dc3d-43c4-4899-b605-4579c70a1dca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:17:02.926318 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.926294    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f81dc3d-43c4-4899-b605-4579c70a1dca-kube-api-access-9hvwz" (OuterVolumeSpecName: "kube-api-access-9hvwz") pod "8f81dc3d-43c4-4899-b605-4579c70a1dca" (UID: "8f81dc3d-43c4-4899-b605-4579c70a1dca"). InnerVolumeSpecName "kube-api-access-9hvwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:17:02.926403 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:02.926298    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f81dc3d-43c4-4899-b605-4579c70a1dca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f81dc3d-43c4-4899-b605-4579c70a1dca" (UID: "8f81dc3d-43c4-4899-b605-4579c70a1dca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:17:03.025223 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.025196    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hvwz\" (UniqueName: \"kubernetes.io/projected/8f81dc3d-43c4-4899-b605-4579c70a1dca-kube-api-access-9hvwz\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:17:03.025223 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.025222    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f81dc3d-43c4-4899-b605-4579c70a1dca-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:17:03.025402 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.025232    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f81dc3d-43c4-4899-b605-4579c70a1dca-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:17:03.351142 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.351027    2580 generic.go:356] "Generic (PLEG): container finished" podID="8f81dc3d-43c4-4899-b605-4579c70a1dca" containerID="ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1" exitCode=0
Jul 18 02:17:03.351142 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.351073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv" event={"ID":"8f81dc3d-43c4-4899-b605-4579c70a1dca","Type":"ContainerDied","Data":"ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1"}
Jul 18 02:17:03.351142 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.351089    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"
Jul 18 02:17:03.351418 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.351157    2580 scope.go:117] "RemoveContainer" containerID="ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1"
Jul 18 02:17:03.351418 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.351143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv" event={"ID":"8f81dc3d-43c4-4899-b605-4579c70a1dca","Type":"ContainerDied","Data":"bd99cfa0b8f8b479fbc1d9b76d3a8f8e2cc085e0538ce596ed8f71fea3adc807"}
Jul 18 02:17:03.360367 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.360348    2580 scope.go:117] "RemoveContainer" containerID="ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1"
Jul 18 02:17:03.360610 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:17:03.360589    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1\": container with ID starting with ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1 not found: ID does not exist" containerID="ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1"
Jul 18 02:17:03.360677 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.360622    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1"} err="failed to get container status \"ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1\": rpc error: code = NotFound desc = could not find container \"ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1\": container with ID starting with ff8626bf8b531b830e2143156ca5ccb6639fc0fb7dc320bfd672b4b1b20b7ea1 not found: ID does not exist"
Jul 18 02:17:03.372290 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.372269    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"]
Jul 18 02:17:03.374388 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.374367    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9a88968c-6ff78799b9-fpfcv"]
Jul 18 02:17:03.390180 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:03.390156    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f81dc3d-43c4-4899-b605-4579c70a1dca" path="/var/lib/kubelet/pods/8f81dc3d-43c4-4899-b605-4579c70a1dca/volumes"
Jul 18 02:17:07.256897 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:07.256851    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"]
Jul 18 02:17:07.258754 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:07.258729    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9a88968c-49l86"]
Jul 18 02:17:07.259779 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:07.259752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:07.390877 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:07.390844    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c46def0b-aa3d-414b-bc34-2c7da272a537" path="/var/lib/kubelet/pods/c46def0b-aa3d-414b-bc34-2c7da272a537/volumes"
Jul 18 02:17:08.433511 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:08.433480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:08.658899 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:08.658851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:08.887243 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:08.887154    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:09.231438 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:09.231406    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:09.345813 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:09.345783    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:11.699243 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.699207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:11.801883 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.801852    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:11.912250 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.912210    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"]
Jul 18 02:17:11.912585 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.912572    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c46def0b-aa3d-414b-bc34-2c7da272a537" containerName="revoke-keys"
Jul 18 02:17:11.912634 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.912588    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c46def0b-aa3d-414b-bc34-2c7da272a537" containerName="revoke-keys"
Jul 18 02:17:11.912634 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.912601    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f81dc3d-43c4-4899-b605-4579c70a1dca" containerName="maas-api"
Jul 18 02:17:11.912634 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.912606    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f81dc3d-43c4-4899-b605-4579c70a1dca" containerName="maas-api"
Jul 18 02:17:11.912731 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.912676    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c46def0b-aa3d-414b-bc34-2c7da272a537" containerName="revoke-keys"
Jul 18 02:17:11.912731 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.912684    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f81dc3d-43c4-4899-b605-4579c70a1dca" containerName="maas-api"
Jul 18 02:17:11.917163 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.917139    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:11.919954 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.919932    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:17:11.921096 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.921079    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-05505cc1-serving-cert\""
Jul 18 02:17:11.921183 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.921100    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:17:11.921254 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.921083    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:17:11.927745 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.927722    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"]
Jul 18 02:17:11.944541 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:11.944512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:12.009919 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.009836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c59735-bfab-49cd-9758-a4100b9152e7-maas-api-tls\") pod \"maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.009919 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.009910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmc8f\" (UniqueName: \"kubernetes.io/projected/b2c59735-bfab-49cd-9758-a4100b9152e7-kube-api-access-kmc8f\") pod \"maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.010144 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.009964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c59735-bfab-49cd-9758-a4100b9152e7-openshift-service-ca\") pod \"maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.017966 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.017925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:12.110414 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.110382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmc8f\" (UniqueName: \"kubernetes.io/projected/b2c59735-bfab-49cd-9758-a4100b9152e7-kube-api-access-kmc8f\") pod \"maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.110572 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.110460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c59735-bfab-49cd-9758-a4100b9152e7-openshift-service-ca\") pod \"maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.110572 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.110484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c59735-bfab-49cd-9758-a4100b9152e7-maas-api-tls\") pod \"maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.111248 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.111221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c59735-bfab-49cd-9758-a4100b9152e7-openshift-service-ca\") pod \"maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.113106 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.113080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c59735-bfab-49cd-9758-a4100b9152e7-maas-api-tls\") pod \"maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.121411 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.121388    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmc8f\" (UniqueName: \"kubernetes.io/projected/b2c59735-bfab-49cd-9758-a4100b9152e7-kube-api-access-kmc8f\") pod \"maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.228625 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.228587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:12.360338 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.360314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"]
Jul 18 02:17:12.362746 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:17:12.362715    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c59735_bfab_49cd_9758_a4100b9152e7.slice/crio-7f9ec41b9242603fd9601c58ed54d2bddc0f1673eb00adad5fed23d9097b12e5 WatchSource:0}: Error finding container 7f9ec41b9242603fd9601c58ed54d2bddc0f1673eb00adad5fed23d9097b12e5: Status 404 returned error can't find the container with id 7f9ec41b9242603fd9601c58ed54d2bddc0f1673eb00adad5fed23d9097b12e5
Jul 18 02:17:12.398003 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.397969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg" event={"ID":"b2c59735-bfab-49cd-9758-a4100b9152e7","Type":"ContainerStarted","Data":"7f9ec41b9242603fd9601c58ed54d2bddc0f1673eb00adad5fed23d9097b12e5"}
Jul 18 02:17:12.566182 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.566096    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:12.666420 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:12.666387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:13.404117 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:13.404080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg" event={"ID":"b2c59735-bfab-49cd-9758-a4100b9152e7","Type":"ContainerStarted","Data":"d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1"}
Jul 18 02:17:13.404572 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:13.404172    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:13.423417 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:13.423370    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg" podStartSLOduration=2.114363855 podStartE2EDuration="2.423354461s" totalImagesPullingTime="308.990606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:17:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:17:13.420610177 +0000 UTC m=+1726.630324399" watchObservedRunningTime="2026-07-18 02:17:13.423354461 +0000 UTC m=+1726.633068680"
Jul 18 02:17:19.415288 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:19.415256    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:36.559054 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:36.559019    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"]
Jul 18 02:17:36.563613 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:36.563593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"
Jul 18 02:17:36.566277 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:36.566261    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:17:36.570105 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:36.570079    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"]
Jul 18 02:17:36.630665 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:36.630634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c6df84-9d9a-4a42-a942-681f72be3a26-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5\" (UID: \"b3c6df84-9d9a-4a42-a942-681f72be3a26\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"
Jul 18 02:17:36.731669 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:36.731633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c6df84-9d9a-4a42-a942-681f72be3a26-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5\" (UID: \"b3c6df84-9d9a-4a42-a942-681f72be3a26\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"
Jul 18 02:17:36.732314 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:36.732293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c6df84-9d9a-4a42-a942-681f72be3a26-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5\" (UID: \"b3c6df84-9d9a-4a42-a942-681f72be3a26\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"
Jul 18 02:17:36.874331 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:36.874250    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"
Jul 18 02:17:37.004503 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:37.004481    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"]
Jul 18 02:17:37.006821 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:17:37.006785    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c6df84_9d9a_4a42_a942_681f72be3a26.slice/crio-d4927a6fcc45d570372c5a6d45b79fd747b15f0ef0039f50d15620e4301e8adf WatchSource:0}: Error finding container d4927a6fcc45d570372c5a6d45b79fd747b15f0ef0039f50d15620e4301e8adf: Status 404 returned error can't find the container with id d4927a6fcc45d570372c5a6d45b79fd747b15f0ef0039f50d15620e4301e8adf
Jul 18 02:17:37.520323 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:37.520228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5" event={"ID":"b3c6df84-9d9a-4a42-a942-681f72be3a26","Type":"ContainerStarted","Data":"1b2ffa3a6c611683a9056e6e6366c0bd475facc6a958b17d47612a181fd9f3d1"}
Jul 18 02:17:37.520323 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:37.520277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5" event={"ID":"b3c6df84-9d9a-4a42-a942-681f72be3a26","Type":"ContainerStarted","Data":"d4927a6fcc45d570372c5a6d45b79fd747b15f0ef0039f50d15620e4301e8adf"}
Jul 18 02:17:37.535930 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:37.535886    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5" podStartSLOduration=1.535871934 podStartE2EDuration="1.535871934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:17:37.534977949 +0000 UTC m=+1750.744692170" watchObservedRunningTime="2026-07-18 02:17:37.535871934 +0000 UTC m=+1750.745586154"
Jul 18 02:17:38.525983 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:38.525949    2580 generic.go:356] "Generic (PLEG): container finished" podID="b3c6df84-9d9a-4a42-a942-681f72be3a26" containerID="1b2ffa3a6c611683a9056e6e6366c0bd475facc6a958b17d47612a181fd9f3d1" exitCode=0
Jul 18 02:17:38.526378 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:38.526090    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5" event={"ID":"b3c6df84-9d9a-4a42-a942-681f72be3a26","Type":"ContainerDied","Data":"1b2ffa3a6c611683a9056e6e6366c0bd475facc6a958b17d47612a181fd9f3d1"}
Jul 18 02:17:39.661990 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:39.661967    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"
Jul 18 02:17:39.758177 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:39.758146    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c6df84-9d9a-4a42-a942-681f72be3a26-maas-api-service-ca\") pod \"b3c6df84-9d9a-4a42-a942-681f72be3a26\" (UID: \"b3c6df84-9d9a-4a42-a942-681f72be3a26\") "
Jul 18 02:17:39.758536 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:39.758510    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c6df84-9d9a-4a42-a942-681f72be3a26-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3c6df84-9d9a-4a42-a942-681f72be3a26" (UID: "b3c6df84-9d9a-4a42-a942-681f72be3a26"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:17:39.859757 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:39.859674    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c6df84-9d9a-4a42-a942-681f72be3a26-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:17:40.536524 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:40.536493    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5" event={"ID":"b3c6df84-9d9a-4a42-a942-681f72be3a26","Type":"ContainerDied","Data":"d4927a6fcc45d570372c5a6d45b79fd747b15f0ef0039f50d15620e4301e8adf"}
Jul 18 02:17:40.536713 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:40.536532    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"
Jul 18 02:17:40.536713 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:40.536540    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4927a6fcc45d570372c5a6d45b79fd747b15f0ef0039f50d15620e4301e8adf"
Jul 18 02:17:46.954331 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:46.954295    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"]
Jul 18 02:17:46.954773 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:46.954550    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg" podUID="b2c59735-bfab-49cd-9758-a4100b9152e7" containerName="maas-api" containerID="cri-o://d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1" gracePeriod=30
Jul 18 02:17:46.978037 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:46.978007    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:47.203160 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.203132    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:47.330191 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.330111    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmc8f\" (UniqueName: \"kubernetes.io/projected/b2c59735-bfab-49cd-9758-a4100b9152e7-kube-api-access-kmc8f\") pod \"b2c59735-bfab-49cd-9758-a4100b9152e7\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") "
Jul 18 02:17:47.330191 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.330159    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c59735-bfab-49cd-9758-a4100b9152e7-openshift-service-ca\") pod \"b2c59735-bfab-49cd-9758-a4100b9152e7\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") "
Jul 18 02:17:47.330400 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.330257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c59735-bfab-49cd-9758-a4100b9152e7-maas-api-tls\") pod \"b2c59735-bfab-49cd-9758-a4100b9152e7\" (UID: \"b2c59735-bfab-49cd-9758-a4100b9152e7\") "
Jul 18 02:17:47.330590 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.330564    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2c59735-bfab-49cd-9758-a4100b9152e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b2c59735-bfab-49cd-9758-a4100b9152e7" (UID: "b2c59735-bfab-49cd-9758-a4100b9152e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:17:47.332424 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.332400    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c59735-bfab-49cd-9758-a4100b9152e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b2c59735-bfab-49cd-9758-a4100b9152e7" (UID: "b2c59735-bfab-49cd-9758-a4100b9152e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:17:47.332507 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.332445    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c59735-bfab-49cd-9758-a4100b9152e7-kube-api-access-kmc8f" (OuterVolumeSpecName: "kube-api-access-kmc8f") pod "b2c59735-bfab-49cd-9758-a4100b9152e7" (UID: "b2c59735-bfab-49cd-9758-a4100b9152e7"). InnerVolumeSpecName "kube-api-access-kmc8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:17:47.431519 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.431495    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c59735-bfab-49cd-9758-a4100b9152e7-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:17:47.431519 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.431520    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmc8f\" (UniqueName: \"kubernetes.io/projected/b2c59735-bfab-49cd-9758-a4100b9152e7-kube-api-access-kmc8f\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:17:47.431693 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.431530    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c59735-bfab-49cd-9758-a4100b9152e7-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:17:47.571788 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.571753    2580 generic.go:356] "Generic (PLEG): container finished" podID="b2c59735-bfab-49cd-9758-a4100b9152e7" containerID="d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1" exitCode=0
Jul 18 02:17:47.571966 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.571797    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"
Jul 18 02:17:47.571966 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.571832    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg" event={"ID":"b2c59735-bfab-49cd-9758-a4100b9152e7","Type":"ContainerDied","Data":"d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1"}
Jul 18 02:17:47.571966 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.571885    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg" event={"ID":"b2c59735-bfab-49cd-9758-a4100b9152e7","Type":"ContainerDied","Data":"7f9ec41b9242603fd9601c58ed54d2bddc0f1673eb00adad5fed23d9097b12e5"}
Jul 18 02:17:47.571966 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.571907    2580 scope.go:117] "RemoveContainer" containerID="d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1"
Jul 18 02:17:47.580570 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.580550    2580 scope.go:117] "RemoveContainer" containerID="d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1"
Jul 18 02:17:47.580848 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:17:47.580829    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1\": container with ID starting with d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1 not found: ID does not exist" containerID="d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1"
Jul 18 02:17:47.580904 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.580858    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1"} err="failed to get container status \"d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1\": rpc error: code = NotFound desc = could not find container \"d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1\": container with ID starting with d6fc455d4b7eb7d0757ed795363945b9eed51038c332e3c693f39605d1a578b1 not found: ID does not exist"
Jul 18 02:17:47.596812 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.596783    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"]
Jul 18 02:17:47.603785 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:47.603763    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-05505cc1-685d4d8cd4-v4qwg"]
Jul 18 02:17:49.390542 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:49.390500    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2c59735-bfab-49cd-9758-a4100b9152e7" path="/var/lib/kubelet/pods/b2c59735-bfab-49cd-9758-a4100b9152e7/volumes"
Jul 18 02:17:51.624474 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:51.624438    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"]
Jul 18 02:17:51.627106 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:51.627075    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:51.627934 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:51.627916    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-05505cc1-w9zb5"]
Jul 18 02:17:51.782421 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:51.782391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:51.998585 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:51.998546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:52.449613 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:52.449573    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:52.547639 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:52.547605    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:52.648018 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:52.647983    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:53.390779 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:53.390747    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3c6df84-9d9a-4a42-a942-681f72be3a26" path="/var/lib/kubelet/pods/b3c6df84-9d9a-4a42-a942-681f72be3a26/volumes"
Jul 18 02:17:53.923928 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:53.923888    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:54.036449 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:54.036417    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:54.133776 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:17:54.133743    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:00.190471 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:00.190426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:00.251504 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:00.251470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:00.341733 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:00.341699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:00.441475 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:00.441381    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:00.544422 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:00.544382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:00.923019 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:00.922983    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:15.941843 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:15.941802    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:16.044904 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:16.044839    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:26.856309 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:26.856261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:33.320567 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:33.320518    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:33.422087 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:33.422030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:33.521586 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:33.521536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:49.436971 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:49.436935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:55.228961 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:55.228889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:55.344440 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:55.344406    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:55.432450 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:18:55.432411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:05.936490 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:05.936446    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:06.760593 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:06.760553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:13.557436 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:13.557395    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:13.671879 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:13.671848    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:13.783334 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:13.783305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:13.876865 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:13.876775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:14.691749 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:14.691696    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:14.810566 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:14.810530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:14.945410 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:14.945309    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:16.666563 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:16.666519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:22.844569 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:22.844528    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:22.945166 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:22.945125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:23.560908 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:23.560864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:30.043435 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:30.043389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:30.143521 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:30.143491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:30.309837 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:30.309742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:30.643127 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:30.643021    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:30.847688 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:30.847651    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:34.166551 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.166514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:34.250887 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.250851    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"]
Jul 18 02:19:34.251239 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.251227    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c59735-bfab-49cd-9758-a4100b9152e7" containerName="maas-api"
Jul 18 02:19:34.251297 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.251241    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c59735-bfab-49cd-9758-a4100b9152e7" containerName="maas-api"
Jul 18 02:19:34.251297 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.251258    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3c6df84-9d9a-4a42-a942-681f72be3a26" containerName="revoke-keys"
Jul 18 02:19:34.251297 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.251264    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3c6df84-9d9a-4a42-a942-681f72be3a26" containerName="revoke-keys"
Jul 18 02:19:34.251391 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.251326    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3c6df84-9d9a-4a42-a942-681f72be3a26" containerName="revoke-keys"
Jul 18 02:19:34.251391 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.251337    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c59735-bfab-49cd-9758-a4100b9152e7" containerName="maas-api"
Jul 18 02:19:34.254275 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.254258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.256906 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.256877    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b9dbc17d-serving-cert\""
Jul 18 02:19:34.257290 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.257270    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:19:34.260491 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.260463    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:34.260759 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.260739    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:19:34.260854 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.260762    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:19:34.266318 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.266293    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"]
Jul 18 02:19:34.333889 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.333851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:34.364057 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.364018    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78570de8-fefe-4e65-a204-6ba2b12e8344-openshift-service-ca\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.364225 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.364113    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78570de8-fefe-4e65-a204-6ba2b12e8344-maas-api-tls\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.364276 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.364231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wslj\" (UniqueName: \"kubernetes.io/projected/78570de8-fefe-4e65-a204-6ba2b12e8344-kube-api-access-9wslj\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.465425 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.465342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wslj\" (UniqueName: \"kubernetes.io/projected/78570de8-fefe-4e65-a204-6ba2b12e8344-kube-api-access-9wslj\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.465425 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.465384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78570de8-fefe-4e65-a204-6ba2b12e8344-openshift-service-ca\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.465645 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.465456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78570de8-fefe-4e65-a204-6ba2b12e8344-maas-api-tls\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.465645 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:19:34.465553    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-serving-cert: secret "maas-api-e2e-mt-b9dbc17d-serving-cert" not found
Jul 18 02:19:34.465645 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:19:34.465614    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78570de8-fefe-4e65-a204-6ba2b12e8344-maas-api-tls podName:78570de8-fefe-4e65-a204-6ba2b12e8344 nodeName:}" failed. No retries permitted until 2026-07-18 02:19:34.965596422 +0000 UTC m=+1868.175310620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/78570de8-fefe-4e65-a204-6ba2b12e8344-maas-api-tls") pod "maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd" (UID: "78570de8-fefe-4e65-a204-6ba2b12e8344") : secret "maas-api-e2e-mt-b9dbc17d-serving-cert" not found
Jul 18 02:19:34.465991 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.465972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78570de8-fefe-4e65-a204-6ba2b12e8344-openshift-service-ca\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.484927 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.484898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wslj\" (UniqueName: \"kubernetes.io/projected/78570de8-fefe-4e65-a204-6ba2b12e8344-kube-api-access-9wslj\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.680741 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.680706    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:34.932873 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.932841    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:34.970460 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.970426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78570de8-fefe-4e65-a204-6ba2b12e8344-maas-api-tls\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:34.972934 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:34.972900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78570de8-fefe-4e65-a204-6ba2b12e8344-maas-api-tls\") pod \"maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:35.033084 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:35.033028    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:35.165539 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:35.165508    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:35.298475 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:35.298388    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"]
Jul 18 02:19:35.300986 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:19:35.300955    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78570de8_fefe_4e65_a204_6ba2b12e8344.slice/crio-3ccbb7b7be9916423de6f23b605f8300063b63a178bdb6a21deff5adaf04a422 WatchSource:0}: Error finding container 3ccbb7b7be9916423de6f23b605f8300063b63a178bdb6a21deff5adaf04a422: Status 404 returned error can't find the container with id 3ccbb7b7be9916423de6f23b605f8300063b63a178bdb6a21deff5adaf04a422
Jul 18 02:19:36.092815 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:36.092723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd" event={"ID":"78570de8-fefe-4e65-a204-6ba2b12e8344","Type":"ContainerStarted","Data":"917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594"}
Jul 18 02:19:36.092815 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:36.092783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd" event={"ID":"78570de8-fefe-4e65-a204-6ba2b12e8344","Type":"ContainerStarted","Data":"3ccbb7b7be9916423de6f23b605f8300063b63a178bdb6a21deff5adaf04a422"}
Jul 18 02:19:36.092815 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:36.092816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:36.113878 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:36.113833    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd" podStartSLOduration=1.6895680419999999 podStartE2EDuration="2.113817857s" totalImagesPullingTime="424.249815ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:19:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:19:36.111559939 +0000 UTC m=+1869.321274160" watchObservedRunningTime="2026-07-18 02:19:36.113817857 +0000 UTC m=+1869.323532113"
Jul 18 02:19:38.711666 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:38.711628    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:39.439317 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:39.439281    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:39.530948 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:39.530913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:39.897981 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:39.897865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:42.102638 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:42.102608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:45.488529 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:45.488480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:45.590236 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:45.590204    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:45.834668 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:45.834585    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"]
Jul 18 02:19:45.837929 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:45.837911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"
Jul 18 02:19:45.840446 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:45.840425    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:19:45.845816 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:45.845790    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"]
Jul 18 02:19:45.975751 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:45.975715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cf2c94-1740-4f6d-ad0c-eba9d9e50991-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq\" (UID: \"81cf2c94-1740-4f6d-ad0c-eba9d9e50991\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"
Jul 18 02:19:46.077131 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:46.077098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cf2c94-1740-4f6d-ad0c-eba9d9e50991-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq\" (UID: \"81cf2c94-1740-4f6d-ad0c-eba9d9e50991\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"
Jul 18 02:19:46.077718 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:46.077698    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cf2c94-1740-4f6d-ad0c-eba9d9e50991-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq\" (UID: \"81cf2c94-1740-4f6d-ad0c-eba9d9e50991\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"
Jul 18 02:19:46.149063 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:46.148953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"
Jul 18 02:19:46.284602 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:46.284574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"]
Jul 18 02:19:46.286984 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:19:46.286942    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81cf2c94_1740_4f6d_ad0c_eba9d9e50991.slice/crio-480316be2c417db76775dec4a20b9b23642ebbf0ead5ed8f0bc66c192bb78c65 WatchSource:0}: Error finding container 480316be2c417db76775dec4a20b9b23642ebbf0ead5ed8f0bc66c192bb78c65: Status 404 returned error can't find the container with id 480316be2c417db76775dec4a20b9b23642ebbf0ead5ed8f0bc66c192bb78c65
Jul 18 02:19:47.146519 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:47.146479    2580 generic.go:356] "Generic (PLEG): container finished" podID="81cf2c94-1740-4f6d-ad0c-eba9d9e50991" containerID="9b7dd646c48fb3cb5963038935075bf02827e95523ebbc2bed29dcf867b9e8ff" exitCode=0
Jul 18 02:19:47.146869 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:47.146575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq" event={"ID":"81cf2c94-1740-4f6d-ad0c-eba9d9e50991","Type":"ContainerDied","Data":"9b7dd646c48fb3cb5963038935075bf02827e95523ebbc2bed29dcf867b9e8ff"}
Jul 18 02:19:47.146869 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:47.146628    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq" event={"ID":"81cf2c94-1740-4f6d-ad0c-eba9d9e50991","Type":"ContainerStarted","Data":"480316be2c417db76775dec4a20b9b23642ebbf0ead5ed8f0bc66c192bb78c65"}
Jul 18 02:19:48.285654 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:48.285626    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"
Jul 18 02:19:48.297010 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:48.296980    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cf2c94-1740-4f6d-ad0c-eba9d9e50991-maas-api-service-ca\") pod \"81cf2c94-1740-4f6d-ad0c-eba9d9e50991\" (UID: \"81cf2c94-1740-4f6d-ad0c-eba9d9e50991\") "
Jul 18 02:19:48.297369 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:48.297345    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81cf2c94-1740-4f6d-ad0c-eba9d9e50991-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81cf2c94-1740-4f6d-ad0c-eba9d9e50991" (UID: "81cf2c94-1740-4f6d-ad0c-eba9d9e50991"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:19:48.397783 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:48.397749    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cf2c94-1740-4f6d-ad0c-eba9d9e50991-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:19:49.158008 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:49.157975    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq" event={"ID":"81cf2c94-1740-4f6d-ad0c-eba9d9e50991","Type":"ContainerDied","Data":"480316be2c417db76775dec4a20b9b23642ebbf0ead5ed8f0bc66c192bb78c65"}
Jul 18 02:19:49.158209 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:49.158028    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="480316be2c417db76775dec4a20b9b23642ebbf0ead5ed8f0bc66c192bb78c65"
Jul 18 02:19:49.158209 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:49.157983    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"
Jul 18 02:19:56.197340 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.197304    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"]
Jul 18 02:19:56.198144 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.197556    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd" podUID="78570de8-fefe-4e65-a204-6ba2b12e8344" containerName="maas-api" containerID="cri-o://917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594" gracePeriod=30
Jul 18 02:19:56.227244 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.226973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:56.444465 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.444440    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:56.467285 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.467222    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78570de8-fefe-4e65-a204-6ba2b12e8344-openshift-service-ca\") pod \"78570de8-fefe-4e65-a204-6ba2b12e8344\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") "
Jul 18 02:19:56.467396 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.467290    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wslj\" (UniqueName: \"kubernetes.io/projected/78570de8-fefe-4e65-a204-6ba2b12e8344-kube-api-access-9wslj\") pod \"78570de8-fefe-4e65-a204-6ba2b12e8344\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") "
Jul 18 02:19:56.467396 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.467373    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78570de8-fefe-4e65-a204-6ba2b12e8344-maas-api-tls\") pod \"78570de8-fefe-4e65-a204-6ba2b12e8344\" (UID: \"78570de8-fefe-4e65-a204-6ba2b12e8344\") "
Jul 18 02:19:56.467599 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.467579    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78570de8-fefe-4e65-a204-6ba2b12e8344-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78570de8-fefe-4e65-a204-6ba2b12e8344" (UID: "78570de8-fefe-4e65-a204-6ba2b12e8344"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:19:56.467716 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.467695    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78570de8-fefe-4e65-a204-6ba2b12e8344-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:19:56.469552 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.469525    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78570de8-fefe-4e65-a204-6ba2b12e8344-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78570de8-fefe-4e65-a204-6ba2b12e8344" (UID: "78570de8-fefe-4e65-a204-6ba2b12e8344"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:19:56.469664 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.469591    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78570de8-fefe-4e65-a204-6ba2b12e8344-kube-api-access-9wslj" (OuterVolumeSpecName: "kube-api-access-9wslj") pod "78570de8-fefe-4e65-a204-6ba2b12e8344" (UID: "78570de8-fefe-4e65-a204-6ba2b12e8344"). InnerVolumeSpecName "kube-api-access-9wslj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:19:56.568397 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.568367    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wslj\" (UniqueName: \"kubernetes.io/projected/78570de8-fefe-4e65-a204-6ba2b12e8344-kube-api-access-9wslj\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:19:56.568397 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:56.568397    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78570de8-fefe-4e65-a204-6ba2b12e8344-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:19:57.196264 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.196223    2580 generic.go:356] "Generic (PLEG): container finished" podID="78570de8-fefe-4e65-a204-6ba2b12e8344" containerID="917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594" exitCode=0
Jul 18 02:19:57.196444 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.196271    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"
Jul 18 02:19:57.196444 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.196315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd" event={"ID":"78570de8-fefe-4e65-a204-6ba2b12e8344","Type":"ContainerDied","Data":"917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594"}
Jul 18 02:19:57.196444 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.196397    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd" event={"ID":"78570de8-fefe-4e65-a204-6ba2b12e8344","Type":"ContainerDied","Data":"3ccbb7b7be9916423de6f23b605f8300063b63a178bdb6a21deff5adaf04a422"}
Jul 18 02:19:57.196444 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.196422    2580 scope.go:117] "RemoveContainer" containerID="917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594"
Jul 18 02:19:57.206584 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.206418    2580 scope.go:117] "RemoveContainer" containerID="917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594"
Jul 18 02:19:57.206821 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:19:57.206678    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594\": container with ID starting with 917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594 not found: ID does not exist" containerID="917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594"
Jul 18 02:19:57.206821 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.206706    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594"} err="failed to get container status \"917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594\": rpc error: code = NotFound desc = could not find container \"917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594\": container with ID starting with 917bc29ff5e3e529dc4dba1bf14410f6db98c2497f2136f379fe5c52c08ad594 not found: ID does not exist"
Jul 18 02:19:57.219851 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.219826    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"]
Jul 18 02:19:57.225523 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.225500    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b9dbc17d-54d9fc9968-v6zrd"]
Jul 18 02:19:57.391275 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:19:57.391241    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78570de8-fefe-4e65-a204-6ba2b12e8344" path="/var/lib/kubelet/pods/78570de8-fefe-4e65-a204-6ba2b12e8344/volumes"
Jul 18 02:20:00.905455 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:00.905417    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"]
Jul 18 02:20:00.909194 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:00.909159    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9dbc17d-4hzsq"]
Jul 18 02:20:00.913811 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:00.913758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:01.338019 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:01.337982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:01.390640 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:01.390609    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81cf2c94-1740-4f6d-ad0c-eba9d9e50991" path="/var/lib/kubelet/pods/81cf2c94-1740-4f6d-ad0c-eba9d9e50991/volumes"
Jul 18 02:20:07.148249 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:07.148211    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:07.251287 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:07.251254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:07.428571 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:07.428528    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:07.653652 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:07.653610    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:07.779545 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:07.779448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:07.881684 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:07.881649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:08.006605 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:08.006572    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:13.859774 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:13.859732    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:13.952972 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:13.952930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:14.149524 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:14.149412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:14.225655 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:14.225624    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:14.325373 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:14.325336    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:19.977837 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:19.977799    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:20.078602 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:20.078566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:26.803645 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:26.803602    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:27.112274 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:27.112186    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:27.290935 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:27.290901    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:30.489696 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.489652    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:30.575869 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.575833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:30.588347 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.588319    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"]
Jul 18 02:20:30.588688 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.588677    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78570de8-fefe-4e65-a204-6ba2b12e8344" containerName="maas-api"
Jul 18 02:20:30.588732 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.588690    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78570de8-fefe-4e65-a204-6ba2b12e8344" containerName="maas-api"
Jul 18 02:20:30.588732 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.588697    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81cf2c94-1740-4f6d-ad0c-eba9d9e50991" containerName="revoke-keys"
Jul 18 02:20:30.588732 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.588703    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="81cf2c94-1740-4f6d-ad0c-eba9d9e50991" containerName="revoke-keys"
Jul 18 02:20:30.588828 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.588760    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="81cf2c94-1740-4f6d-ad0c-eba9d9e50991" containerName="revoke-keys"
Jul 18 02:20:30.588828 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.588769    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="78570de8-fefe-4e65-a204-6ba2b12e8344" containerName="maas-api"
Jul 18 02:20:30.591971 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.591949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:30.594838 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.594814    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-546eb0de-serving-cert\""
Jul 18 02:20:30.594980 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.594867    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:20:30.596001 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.595983    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:20:30.596133 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.595985    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:20:30.604907 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.604886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"]
Jul 18 02:20:30.670836 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.670800    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:30.774903 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.774809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b0a128-9171-404f-81a5-912dfa12498c-maas-api-tls\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:30.774903 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.774866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b0a128-9171-404f-81a5-912dfa12498c-openshift-service-ca\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:30.775129 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.774989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfrqh\" (UniqueName: \"kubernetes.io/projected/c2b0a128-9171-404f-81a5-912dfa12498c-kube-api-access-nfrqh\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:30.875915 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.875881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfrqh\" (UniqueName: \"kubernetes.io/projected/c2b0a128-9171-404f-81a5-912dfa12498c-kube-api-access-nfrqh\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:30.876101 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.875925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b0a128-9171-404f-81a5-912dfa12498c-maas-api-tls\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:30.876101 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.875971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b0a128-9171-404f-81a5-912dfa12498c-openshift-service-ca\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:30.876175 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:20:30.876105    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-serving-cert: secret "maas-api-e2e-mt-546eb0de-serving-cert" not found
Jul 18 02:20:30.876221 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:20:30.876189    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b0a128-9171-404f-81a5-912dfa12498c-maas-api-tls podName:c2b0a128-9171-404f-81a5-912dfa12498c nodeName:}" failed. No retries permitted until 2026-07-18 02:20:31.376167759 +0000 UTC m=+1924.585881956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c2b0a128-9171-404f-81a5-912dfa12498c-maas-api-tls") pod "maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl" (UID: "c2b0a128-9171-404f-81a5-912dfa12498c") : secret "maas-api-e2e-mt-546eb0de-serving-cert" not found
Jul 18 02:20:30.876614 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.876596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b0a128-9171-404f-81a5-912dfa12498c-openshift-service-ca\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:30.893195 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:30.893166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfrqh\" (UniqueName: \"kubernetes.io/projected/c2b0a128-9171-404f-81a5-912dfa12498c-kube-api-access-nfrqh\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:31.231580 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:31.231536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:31.331609 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:31.331569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:31.381409 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:31.381380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b0a128-9171-404f-81a5-912dfa12498c-maas-api-tls\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:31.383983 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:31.383964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b0a128-9171-404f-81a5-912dfa12498c-maas-api-tls\") pod \"maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:31.429753 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:31.429725    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:31.502033 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:31.501963    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:31.633355 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:31.633330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"]
Jul 18 02:20:31.636208 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:20:31.636169    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b0a128_9171_404f_81a5_912dfa12498c.slice/crio-718eddfea2c816eff558b25e26fa1a1b8362374411c20497fd6661f762b88b91 WatchSource:0}: Error finding container 718eddfea2c816eff558b25e26fa1a1b8362374411c20497fd6661f762b88b91: Status 404 returned error can't find the container with id 718eddfea2c816eff558b25e26fa1a1b8362374411c20497fd6661f762b88b91
Jul 18 02:20:31.637550 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:31.637529    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:20:32.367190 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:32.367108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl" event={"ID":"c2b0a128-9171-404f-81a5-912dfa12498c","Type":"ContainerStarted","Data":"7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec"}
Jul 18 02:20:32.367190 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:32.367161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl" event={"ID":"c2b0a128-9171-404f-81a5-912dfa12498c","Type":"ContainerStarted","Data":"718eddfea2c816eff558b25e26fa1a1b8362374411c20497fd6661f762b88b91"}
Jul 18 02:20:32.367190 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:32.367189    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:32.383392 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:32.383343    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl" podStartSLOduration=1.9993628860000001 podStartE2EDuration="2.383327671s" totalImagesPullingTime="383.964785ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:20:32.382336019 +0000 UTC m=+1925.592050236" watchObservedRunningTime="2026-07-18 02:20:32.383327671 +0000 UTC m=+1925.593041890"
Jul 18 02:20:34.863747 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:34.863704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:38.380660 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:38.380633    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:20:39.583119 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:39.583083    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:39.683720 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:39.683689    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:42.304518 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.304486    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"]
Jul 18 02:20:42.307774 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.307752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.310525 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.310495    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-546eb0de\"/\"kube-root-ca.crt\""
Jul 18 02:20:42.311904 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.311880    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-546eb0de\"/\"openshift-service-ca.crt\""
Jul 18 02:20:42.312036 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.311914    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-546eb0de\"/\"e2e-lifecycle-model-546eb0de-kserve-self-signed-certs\""
Jul 18 02:20:42.312036 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.311955    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-546eb0de\"/\"default-dockercfg-fz8mw\""
Jul 18 02:20:42.317537 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.317516    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"]
Jul 18 02:20:42.337984 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.337950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:42.371245 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.371223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-model-cache\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.371373 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.371252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-home\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.371373 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.371273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/728922d1-a92b-4efa-8f78-134a8df70fd2-tls-certs\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.371373 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.371305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2rwl\" (UniqueName: \"kubernetes.io/projected/728922d1-a92b-4efa-8f78-134a8df70fd2-kube-api-access-f2rwl\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.371539 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.371472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-dshm\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.371539 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.371509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-tmp-dir\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.472935 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.472903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-dshm\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.472935 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.472940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-tmp-dir\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.473189 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.472973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-model-cache\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.473189 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.472999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-home\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.473189 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.473025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/728922d1-a92b-4efa-8f78-134a8df70fd2-tls-certs\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.473189 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.473102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2rwl\" (UniqueName: \"kubernetes.io/projected/728922d1-a92b-4efa-8f78-134a8df70fd2-kube-api-access-f2rwl\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.473446 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.473424    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-model-cache\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.473521 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.473470    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-tmp-dir\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.473584 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.473544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-home\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.475339 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.475318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-dshm\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.475626 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.475608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/728922d1-a92b-4efa-8f78-134a8df70fd2-tls-certs\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.481254 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.481234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2rwl\" (UniqueName: \"kubernetes.io/projected/728922d1-a92b-4efa-8f78-134a8df70fd2-kube-api-access-f2rwl\") pod \"e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") " pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.618801 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.618714    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:42.753851 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:42.753829    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"]
Jul 18 02:20:42.755847 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:20:42.755810    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod728922d1_a92b_4efa_8f78_134a8df70fd2.slice/crio-9b98bac44bf706beb7efcd787987902b64c448cd793d93fcb823534a8719632d WatchSource:0}: Error finding container 9b98bac44bf706beb7efcd787987902b64c448cd793d93fcb823534a8719632d: Status 404 returned error can't find the container with id 9b98bac44bf706beb7efcd787987902b64c448cd793d93fcb823534a8719632d
Jul 18 02:20:43.421473 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:43.421434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr" event={"ID":"728922d1-a92b-4efa-8f78-134a8df70fd2","Type":"ContainerStarted","Data":"aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9"}
Jul 18 02:20:43.421884 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:43.421480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr" event={"ID":"728922d1-a92b-4efa-8f78-134a8df70fd2","Type":"ContainerStarted","Data":"9b98bac44bf706beb7efcd787987902b64c448cd793d93fcb823534a8719632d"}
Jul 18 02:20:43.439781 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:43.439734    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr" podStartSLOduration=1.439716171 podStartE2EDuration="1.439716171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:20:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:20:43.437837442 +0000 UTC m=+1936.647551666" watchObservedRunningTime="2026-07-18 02:20:43.439716171 +0000 UTC m=+1936.649430391"
Jul 18 02:20:52.619820 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:52.619781    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:52.620294 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:52.619841    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:52.632330 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:52.632307    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:20:53.480162 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:20:53.480128    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:21:03.097228 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:03.097190    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"]
Jul 18 02:21:03.100648 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:03.100630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"
Jul 18 02:21:03.103207 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:03.103188    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:21:03.106844 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:03.106823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"]
Jul 18 02:21:03.164250 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:03.164213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c283dd26-fbed-471c-82d0-8b7667231a27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh\" (UID: \"c283dd26-fbed-471c-82d0-8b7667231a27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"
Jul 18 02:21:03.265374 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:03.265341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c283dd26-fbed-471c-82d0-8b7667231a27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh\" (UID: \"c283dd26-fbed-471c-82d0-8b7667231a27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"
Jul 18 02:21:03.265948 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:03.265930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c283dd26-fbed-471c-82d0-8b7667231a27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh\" (UID: \"c283dd26-fbed-471c-82d0-8b7667231a27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"
Jul 18 02:21:03.413217 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:03.413138    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"
Jul 18 02:21:03.553597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:03.553568    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"]
Jul 18 02:21:03.555890 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:21:03.555860    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc283dd26_fbed_471c_82d0_8b7667231a27.slice/crio-29b101d54c381380b331ec854db8591e104e978c5a4c5bcb9fd2451e4c955f37 WatchSource:0}: Error finding container 29b101d54c381380b331ec854db8591e104e978c5a4c5bcb9fd2451e4c955f37: Status 404 returned error can't find the container with id 29b101d54c381380b331ec854db8591e104e978c5a4c5bcb9fd2451e4c955f37
Jul 18 02:21:04.524822 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:04.524735    2580 generic.go:356] "Generic (PLEG): container finished" podID="c283dd26-fbed-471c-82d0-8b7667231a27" containerID="510d26cf283a4d12d617b9bb9fab13786e733b132ea9b231329686c91fb1ea07" exitCode=0
Jul 18 02:21:04.525201 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:04.524830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh" event={"ID":"c283dd26-fbed-471c-82d0-8b7667231a27","Type":"ContainerDied","Data":"510d26cf283a4d12d617b9bb9fab13786e733b132ea9b231329686c91fb1ea07"}
Jul 18 02:21:04.525201 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:04.524884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh" event={"ID":"c283dd26-fbed-471c-82d0-8b7667231a27","Type":"ContainerStarted","Data":"29b101d54c381380b331ec854db8591e104e978c5a4c5bcb9fd2451e4c955f37"}
Jul 18 02:21:05.656629 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:05.656602    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"
Jul 18 02:21:05.789889 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:05.789795    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c283dd26-fbed-471c-82d0-8b7667231a27-maas-api-service-ca\") pod \"c283dd26-fbed-471c-82d0-8b7667231a27\" (UID: \"c283dd26-fbed-471c-82d0-8b7667231a27\") "
Jul 18 02:21:05.790203 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:05.790179    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c283dd26-fbed-471c-82d0-8b7667231a27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c283dd26-fbed-471c-82d0-8b7667231a27" (UID: "c283dd26-fbed-471c-82d0-8b7667231a27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:21:05.890761 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:05.890732    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c283dd26-fbed-471c-82d0-8b7667231a27-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:06.537941 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:06.537915    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"
Jul 18 02:21:06.538119 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:06.537947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh" event={"ID":"c283dd26-fbed-471c-82d0-8b7667231a27","Type":"ContainerDied","Data":"29b101d54c381380b331ec854db8591e104e978c5a4c5bcb9fd2451e4c955f37"}
Jul 18 02:21:06.538119 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:06.537994    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29b101d54c381380b331ec854db8591e104e978c5a4c5bcb9fd2451e4c955f37"
Jul 18 02:21:18.720626 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:18.717655    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"]
Jul 18 02:21:18.720626 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:18.717992    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl" podUID="c2b0a128-9171-404f-81a5-912dfa12498c" containerName="maas-api" containerID="cri-o://7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec" gracePeriod=30
Jul 18 02:21:18.750193 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:18.750156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:18.962948 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:18.962925    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:21:19.107346 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.107257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b0a128-9171-404f-81a5-912dfa12498c-openshift-service-ca\") pod \"c2b0a128-9171-404f-81a5-912dfa12498c\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") "
Jul 18 02:21:19.107346 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.107304    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b0a128-9171-404f-81a5-912dfa12498c-maas-api-tls\") pod \"c2b0a128-9171-404f-81a5-912dfa12498c\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") "
Jul 18 02:21:19.107346 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.107322    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfrqh\" (UniqueName: \"kubernetes.io/projected/c2b0a128-9171-404f-81a5-912dfa12498c-kube-api-access-nfrqh\") pod \"c2b0a128-9171-404f-81a5-912dfa12498c\" (UID: \"c2b0a128-9171-404f-81a5-912dfa12498c\") "
Jul 18 02:21:19.107647 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.107627    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2b0a128-9171-404f-81a5-912dfa12498c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c2b0a128-9171-404f-81a5-912dfa12498c" (UID: "c2b0a128-9171-404f-81a5-912dfa12498c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:21:19.109674 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.109639    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2b0a128-9171-404f-81a5-912dfa12498c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c2b0a128-9171-404f-81a5-912dfa12498c" (UID: "c2b0a128-9171-404f-81a5-912dfa12498c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:21:19.109782 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.109642    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2b0a128-9171-404f-81a5-912dfa12498c-kube-api-access-nfrqh" (OuterVolumeSpecName: "kube-api-access-nfrqh") pod "c2b0a128-9171-404f-81a5-912dfa12498c" (UID: "c2b0a128-9171-404f-81a5-912dfa12498c"). InnerVolumeSpecName "kube-api-access-nfrqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:21:19.208339 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.208308    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b0a128-9171-404f-81a5-912dfa12498c-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:19.208339 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.208337    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b0a128-9171-404f-81a5-912dfa12498c-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:19.208339 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.208346    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfrqh\" (UniqueName: \"kubernetes.io/projected/c2b0a128-9171-404f-81a5-912dfa12498c-kube-api-access-nfrqh\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:19.602399 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.602368    2580 generic.go:356] "Generic (PLEG): container finished" podID="c2b0a128-9171-404f-81a5-912dfa12498c" containerID="7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec" exitCode=0
Jul 18 02:21:19.602583 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.602418    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"
Jul 18 02:21:19.602583 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.602441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl" event={"ID":"c2b0a128-9171-404f-81a5-912dfa12498c","Type":"ContainerDied","Data":"7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec"}
Jul 18 02:21:19.602583 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.602495    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl" event={"ID":"c2b0a128-9171-404f-81a5-912dfa12498c","Type":"ContainerDied","Data":"718eddfea2c816eff558b25e26fa1a1b8362374411c20497fd6661f762b88b91"}
Jul 18 02:21:19.602583 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.602517    2580 scope.go:117] "RemoveContainer" containerID="7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec"
Jul 18 02:21:19.611326 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.611305    2580 scope.go:117] "RemoveContainer" containerID="7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec"
Jul 18 02:21:19.611589 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:21:19.611563    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec\": container with ID starting with 7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec not found: ID does not exist" containerID="7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec"
Jul 18 02:21:19.611700 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.611597    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec"} err="failed to get container status \"7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec\": rpc error: code = NotFound desc = could not find container \"7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec\": container with ID starting with 7e968b3d6521c82dc203c9a5e7054b1ac7ba5c525950636cc3c36efa72224bec not found: ID does not exist"
Jul 18 02:21:19.620084 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.620059    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"]
Jul 18 02:21:19.623801 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:19.623778    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546eb0de-6cc45765cc-ns4nl"]
Jul 18 02:21:21.393420 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:21.393387    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2b0a128-9171-404f-81a5-912dfa12498c" path="/var/lib/kubelet/pods/c2b0a128-9171-404f-81a5-912dfa12498c/volumes"
Jul 18 02:21:23.163013 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:23.162981    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"]
Jul 18 02:21:23.165689 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:23.165664    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-546eb0de-xbjxh"]
Jul 18 02:21:23.167881 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:23.167860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:23.391316 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:23.391286    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c283dd26-fbed-471c-82d0-8b7667231a27" path="/var/lib/kubelet/pods/c283dd26-fbed-471c-82d0-8b7667231a27/volumes"
Jul 18 02:21:25.366353 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:25.366317    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:27.440669 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:27.440638    2580 scope.go:117] "RemoveContainer" containerID="b01e6e457ef4e74dd1a6ebf73c52025a56a8f3026f4db6cb12bc7245a8094946"
Jul 18 02:21:30.350486 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:30.350448    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"]
Jul 18 02:21:30.393326 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:30.393277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:45.371232 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.371172    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr" podUID="728922d1-a92b-4efa-8f78-134a8df70fd2" containerName="main" containerID="cri-o://aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9" gracePeriod=45
Jul 18 02:21:45.634729 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.634671    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:21:45.730036 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730003    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-model-cache\") pod \"728922d1-a92b-4efa-8f78-134a8df70fd2\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") "
Jul 18 02:21:45.730241 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730069    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/728922d1-a92b-4efa-8f78-134a8df70fd2-tls-certs\") pod \"728922d1-a92b-4efa-8f78-134a8df70fd2\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") "
Jul 18 02:21:45.730241 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730110    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-tmp-dir\") pod \"728922d1-a92b-4efa-8f78-134a8df70fd2\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") "
Jul 18 02:21:45.730241 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730137    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-home\") pod \"728922d1-a92b-4efa-8f78-134a8df70fd2\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") "
Jul 18 02:21:45.730241 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730167    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-dshm\") pod \"728922d1-a92b-4efa-8f78-134a8df70fd2\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") "
Jul 18 02:21:45.730241 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730233    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2rwl\" (UniqueName: \"kubernetes.io/projected/728922d1-a92b-4efa-8f78-134a8df70fd2-kube-api-access-f2rwl\") pod \"728922d1-a92b-4efa-8f78-134a8df70fd2\" (UID: \"728922d1-a92b-4efa-8f78-134a8df70fd2\") "
Jul 18 02:21:45.730522 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-model-cache" (OuterVolumeSpecName: "model-cache") pod "728922d1-a92b-4efa-8f78-134a8df70fd2" (UID: "728922d1-a92b-4efa-8f78-134a8df70fd2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:21:45.730522 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730344    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "728922d1-a92b-4efa-8f78-134a8df70fd2" (UID: "728922d1-a92b-4efa-8f78-134a8df70fd2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:21:45.730522 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730401    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-home" (OuterVolumeSpecName: "home") pod "728922d1-a92b-4efa-8f78-134a8df70fd2" (UID: "728922d1-a92b-4efa-8f78-134a8df70fd2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:21:45.730668 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730565    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-model-cache\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:45.730668 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730609    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-tmp-dir\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:45.730668 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.730623    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-home\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:45.732485 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.732457    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-dshm" (OuterVolumeSpecName: "dshm") pod "728922d1-a92b-4efa-8f78-134a8df70fd2" (UID: "728922d1-a92b-4efa-8f78-134a8df70fd2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:21:45.732619 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.732482    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/728922d1-a92b-4efa-8f78-134a8df70fd2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "728922d1-a92b-4efa-8f78-134a8df70fd2" (UID: "728922d1-a92b-4efa-8f78-134a8df70fd2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:21:45.732619 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.732604    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/728922d1-a92b-4efa-8f78-134a8df70fd2-kube-api-access-f2rwl" (OuterVolumeSpecName: "kube-api-access-f2rwl") pod "728922d1-a92b-4efa-8f78-134a8df70fd2" (UID: "728922d1-a92b-4efa-8f78-134a8df70fd2"). InnerVolumeSpecName "kube-api-access-f2rwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:21:45.738242 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.738213    2580 generic.go:356] "Generic (PLEG): container finished" podID="728922d1-a92b-4efa-8f78-134a8df70fd2" containerID="aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9" exitCode=0
Jul 18 02:21:45.738347 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.738253    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"
Jul 18 02:21:45.738347 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.738295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr" event={"ID":"728922d1-a92b-4efa-8f78-134a8df70fd2","Type":"ContainerDied","Data":"aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9"}
Jul 18 02:21:45.738347 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.738340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr" event={"ID":"728922d1-a92b-4efa-8f78-134a8df70fd2","Type":"ContainerDied","Data":"9b98bac44bf706beb7efcd787987902b64c448cd793d93fcb823534a8719632d"}
Jul 18 02:21:45.738454 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.738359    2580 scope.go:117] "RemoveContainer" containerID="aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9"
Jul 18 02:21:45.753188 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.753169    2580 scope.go:117] "RemoveContainer" containerID="aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9"
Jul 18 02:21:45.753611 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:21:45.753593    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9\": container with ID starting with aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9 not found: ID does not exist" containerID="aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9"
Jul 18 02:21:45.753674 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.753621    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9"} err="failed to get container status \"aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9\": rpc error: code = NotFound desc = could not find container \"aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9\": container with ID starting with aebef7d571f044d1419617974c7466cab105a729142ba22fbffe9401f8cbe3a9 not found: ID does not exist"
Jul 18 02:21:45.763288 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.763266    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"]
Jul 18 02:21:45.766591 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.766569    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-546eb0de/e2e-lifecycle-model-546eb0de-kserve-57b78dc6d7-n59hr"]
Jul 18 02:21:45.831984 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.831959    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/728922d1-a92b-4efa-8f78-134a8df70fd2-dshm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:45.831984 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.831983    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2rwl\" (UniqueName: \"kubernetes.io/projected/728922d1-a92b-4efa-8f78-134a8df70fd2-kube-api-access-f2rwl\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:45.832140 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:45.831994    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/728922d1-a92b-4efa-8f78-134a8df70fd2-tls-certs\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:21:47.391054 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:21:47.390997    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="728922d1-a92b-4efa-8f78-134a8df70fd2" path="/var/lib/kubelet/pods/728922d1-a92b-4efa-8f78-134a8df70fd2/volumes"
Jul 18 02:22:07.204473 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:07.204423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:08.401129 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:08.401095    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:13.442874 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:13.442830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:13.506055 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:13.506020    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:13.718595 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:13.718505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:13.955961 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:13.955925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:15.018423 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:15.018389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:15.117237 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:15.117190    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:15.218914 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:15.218878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:16.100301 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:16.100261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:16.290335 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:16.290304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:17.748342 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:17.748302    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:23.650248 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:23.650206    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:23.750510 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:23.750479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:24.374133 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:24.374099    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:27.463105 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:27.463073    2580 scope.go:117] "RemoveContainer" containerID="828935eae3b247edf4f8ca873c0cb04aca7fdae0ac6a96a2eb200de62bb1380b"
Jul 18 02:22:31.200297 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:31.200255    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:31.307667 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:31.307631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:31.443792 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:31.443761    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:41.973927 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:41.973892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:42.288330 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:42.288247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:54.762494 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:54.762399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:54.864146 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:22:54.864114    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:03.245977 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:03.245938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:09.610131 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:09.610100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:09.711596 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:09.711544    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:09.956182 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:09.956136    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:10.279616 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:10.279538    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:10.608533 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:10.608431    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:12.446256 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:12.446221    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:13.337450 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.337413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:13.415273 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415243    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"]
Jul 18 02:23:13.415614 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415602    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2b0a128-9171-404f-81a5-912dfa12498c" containerName="maas-api"
Jul 18 02:23:13.415657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415616    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2b0a128-9171-404f-81a5-912dfa12498c" containerName="maas-api"
Jul 18 02:23:13.415657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415624    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="728922d1-a92b-4efa-8f78-134a8df70fd2" containerName="main"
Jul 18 02:23:13.415657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415630    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="728922d1-a92b-4efa-8f78-134a8df70fd2" containerName="main"
Jul 18 02:23:13.415657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415636    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c283dd26-fbed-471c-82d0-8b7667231a27" containerName="revoke-keys"
Jul 18 02:23:13.415657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415642    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c283dd26-fbed-471c-82d0-8b7667231a27" containerName="revoke-keys"
Jul 18 02:23:13.415811 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415693    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c283dd26-fbed-471c-82d0-8b7667231a27" containerName="revoke-keys"
Jul 18 02:23:13.415811 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415703    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="728922d1-a92b-4efa-8f78-134a8df70fd2" containerName="main"
Jul 18 02:23:13.415811 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.415711    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2b0a128-9171-404f-81a5-912dfa12498c" containerName="maas-api"
Jul 18 02:23:13.418577 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.418558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.421387 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.421362    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-8b6a54-serving-cert\""
Jul 18 02:23:13.421387 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.421376    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:23:13.421577 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.421417    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:23:13.422494 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.422477    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:23:13.429684 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.429660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:13.431125 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.431104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"]
Jul 18 02:23:13.492524 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.492488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx6sb\" (UniqueName: \"kubernetes.io/projected/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-kube-api-access-bx6sb\") pod \"maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.492915 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.492542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-maas-api-tls\") pod \"maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.492915 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.492575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.496012 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.495978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:13.593772 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.593680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.593933 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.593781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx6sb\" (UniqueName: \"kubernetes.io/projected/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-kube-api-access-bx6sb\") pod \"maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.593933 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.593854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-maas-api-tls\") pod \"maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.594395 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.594362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.596374 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.596352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-maas-api-tls\") pod \"maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.603236 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.603212    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx6sb\" (UniqueName: \"kubernetes.io/projected/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-kube-api-access-bx6sb\") pod \"maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.728776 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.728736    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:13.862721 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:13.862525    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"]
Jul 18 02:23:13.864830 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:23:13.864790    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247aaa08_fd9f_4ee6_a7d5_15395a5335ff.slice/crio-ee2437cd6c26aa654bc129c212cb9f1b3883dfc73cd35cd3a6883235ef3c3865 WatchSource:0}: Error finding container ee2437cd6c26aa654bc129c212cb9f1b3883dfc73cd35cd3a6883235ef3c3865: Status 404 returned error can't find the container with id ee2437cd6c26aa654bc129c212cb9f1b3883dfc73cd35cd3a6883235ef3c3865
Jul 18 02:23:14.012470 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:14.012436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:14.120109 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:14.120001    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:14.157789 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:14.157752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x" event={"ID":"247aaa08-fd9f-4ee6-a7d5-15395a5335ff","Type":"ContainerStarted","Data":"ee2437cd6c26aa654bc129c212cb9f1b3883dfc73cd35cd3a6883235ef3c3865"}
Jul 18 02:23:15.163789 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:15.163758    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x" event={"ID":"247aaa08-fd9f-4ee6-a7d5-15395a5335ff","Type":"ContainerStarted","Data":"8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221"}
Jul 18 02:23:15.164205 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:15.163868    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:15.185068 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:15.184973    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x" podStartSLOduration=1.7900886329999999 podStartE2EDuration="2.18495967s" totalImagesPullingTime="394.871037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:23:15.184089587 +0000 UTC m=+2088.393803806" watchObservedRunningTime="2026-07-18 02:23:15.18495967 +0000 UTC m=+2088.394673889"
Jul 18 02:23:17.763739 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:17.763695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:18.095206 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:18.095111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:18.389614 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:18.389526    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:18.597312 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:18.597275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:21.043440 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.043407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:21.122153 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.122117    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"]
Jul 18 02:23:21.123525 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.123496    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:21.126253 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.126234    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.128822 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.128759    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f21b76-serving-cert\""
Jul 18 02:23:21.136550 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.136530    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"]
Jul 18 02:23:21.161712 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.161686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992e1281-2bc3-4f17-9676-72cd6bddc526-maas-api-tls\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.161822 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.161744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992e1281-2bc3-4f17-9676-72cd6bddc526-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.161822 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.161805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbhnm\" (UniqueName: \"kubernetes.io/projected/992e1281-2bc3-4f17-9676-72cd6bddc526-kube-api-access-dbhnm\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.173109 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.173090    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:23:21.197475 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.197441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:21.262377 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.262348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992e1281-2bc3-4f17-9676-72cd6bddc526-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.262566 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.262416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbhnm\" (UniqueName: \"kubernetes.io/projected/992e1281-2bc3-4f17-9676-72cd6bddc526-kube-api-access-dbhnm\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.262645 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.262584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992e1281-2bc3-4f17-9676-72cd6bddc526-maas-api-tls\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.262792 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:23:21.262771    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-serving-cert: secret "maas-api-e2e-inf-b-f21b76-serving-cert" not found
Jul 18 02:23:21.262872 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:23:21.262861    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/992e1281-2bc3-4f17-9676-72cd6bddc526-maas-api-tls podName:992e1281-2bc3-4f17-9676-72cd6bddc526 nodeName:}" failed. No retries permitted until 2026-07-18 02:23:21.762835349 +0000 UTC m=+2094.972549550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/992e1281-2bc3-4f17-9676-72cd6bddc526-maas-api-tls") pod "maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z" (UID: "992e1281-2bc3-4f17-9676-72cd6bddc526") : secret "maas-api-e2e-inf-b-f21b76-serving-cert" not found
Jul 18 02:23:21.263102 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.263082    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992e1281-2bc3-4f17-9676-72cd6bddc526-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.272905 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.272881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbhnm\" (UniqueName: \"kubernetes.io/projected/992e1281-2bc3-4f17-9676-72cd6bddc526-kube-api-access-dbhnm\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.323548 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.323471    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:21.768975 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.768936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992e1281-2bc3-4f17-9676-72cd6bddc526-maas-api-tls\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.771541 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.771520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992e1281-2bc3-4f17-9676-72cd6bddc526-maas-api-tls\") pod \"maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:21.847506 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.847466    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:21.947124 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:21.947083    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:22.038263 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:22.038171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:22.168917 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:22.168888    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"]
Jul 18 02:23:22.171118 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:23:22.171090    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod992e1281_2bc3_4f17_9676_72cd6bddc526.slice/crio-bbd7a635bf3760aa624bf93a5a1a24db04de65496f9da5a9488f173ee8541e78 WatchSource:0}: Error finding container bbd7a635bf3760aa624bf93a5a1a24db04de65496f9da5a9488f173ee8541e78: Status 404 returned error can't find the container with id bbd7a635bf3760aa624bf93a5a1a24db04de65496f9da5a9488f173ee8541e78
Jul 18 02:23:22.199304 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:22.199279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z" event={"ID":"992e1281-2bc3-4f17-9676-72cd6bddc526","Type":"ContainerStarted","Data":"bbd7a635bf3760aa624bf93a5a1a24db04de65496f9da5a9488f173ee8541e78"}
Jul 18 02:23:23.205443 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:23.205409    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z" event={"ID":"992e1281-2bc3-4f17-9676-72cd6bddc526","Type":"ContainerStarted","Data":"f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231"}
Jul 18 02:23:23.205852 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:23.205503    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:23.222583 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:23.222534    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z" podStartSLOduration=1.80462131 podStartE2EDuration="2.222518762s" totalImagesPullingTime="417.897452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:23:23.221078652 +0000 UTC m=+2096.430792869" watchObservedRunningTime="2026-07-18 02:23:23.222518762 +0000 UTC m=+2096.432233018"
Jul 18 02:23:25.451289 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:25.451259    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:27.485620 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:27.485590    2580 scope.go:117] "RemoveContainer" containerID="218244cfaf0dfe6d2f628f6660d6a02ba87463643b03d1894ad369bf1f2930ec"
Jul 18 02:23:29.216161 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:29.216131    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:23:34.761830 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:34.761789    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:34.861479 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:34.861424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:37.001758 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.001727    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"]
Jul 18 02:23:37.005327 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.005309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.009661 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.009638    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8b6a54\"/\"default-dockercfg-hzv7k\""
Jul 18 02:23:37.009661 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.009652    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8b6a54\"/\"openshift-service-ca.crt\""
Jul 18 02:23:37.009843 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.009639    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8b6a54\"/\"kube-root-ca.crt\""
Jul 18 02:23:37.009843 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.009638    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8b6a54\"/\"test-model-8b6a54-kserve-self-signed-certs\""
Jul 18 02:23:37.012994 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.012964    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"]
Jul 18 02:23:37.019867 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.019843    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:37.114079 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.114006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tls-certs\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.114276 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.114173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-model-cache\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.114276 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.114214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f4r7\" (UniqueName: \"kubernetes.io/projected/0abb5927-eb48-4efd-8af0-497a44e4b2d7-kube-api-access-6f4r7\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.114276 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.114267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tmp-dir\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.114399 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.114300    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-home\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.114399 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.114321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-dshm\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.215439 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.215401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-model-cache\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.215439 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.215445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f4r7\" (UniqueName: \"kubernetes.io/projected/0abb5927-eb48-4efd-8af0-497a44e4b2d7-kube-api-access-6f4r7\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.215647 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.215578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tmp-dir\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.215647 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.215619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-home\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.215647 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.215642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-dshm\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.215780 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.215705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tls-certs\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.215838 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.215794    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-model-cache\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.215944 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.215923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-home\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.216004 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.215959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tmp-dir\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.217979 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.217954    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-dshm\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.218242 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.218226    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tls-certs\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.223681 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.223652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f4r7\" (UniqueName: \"kubernetes.io/projected/0abb5927-eb48-4efd-8af0-497a44e4b2d7-kube-api-access-6f4r7\") pod \"test-model-8b6a54-kserve-77bb8956d9-k7pzl\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") " pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.315194 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.315102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:37.451957 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:37.451930    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"]
Jul 18 02:23:37.454381 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:23:37.454354    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0abb5927_eb48_4efd_8af0_497a44e4b2d7.slice/crio-264a10114b05f9abf710018a06b319b21a6e2223d670e51a806612b7332e3792 WatchSource:0}: Error finding container 264a10114b05f9abf710018a06b319b21a6e2223d670e51a806612b7332e3792: Status 404 returned error can't find the container with id 264a10114b05f9abf710018a06b319b21a6e2223d670e51a806612b7332e3792
Jul 18 02:23:38.280608 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:38.280571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl" event={"ID":"0abb5927-eb48-4efd-8af0-497a44e4b2d7","Type":"ContainerStarted","Data":"69713760dd2143e6310ab64ab45fe9c35f29651d37c1e0b961919902ce8decea"}
Jul 18 02:23:38.280998 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:38.280614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl" event={"ID":"0abb5927-eb48-4efd-8af0-497a44e4b2d7","Type":"ContainerStarted","Data":"264a10114b05f9abf710018a06b319b21a6e2223d670e51a806612b7332e3792"}
Jul 18 02:23:38.298448 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:38.298396    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl" podStartSLOduration=2.29837936 podStartE2EDuration="2.29837936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:23:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:23:38.296537329 +0000 UTC m=+2111.506251550" watchObservedRunningTime="2026-07-18 02:23:38.29837936 +0000 UTC m=+2111.508093580"
Jul 18 02:23:43.687220 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:43.687168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:43.785460 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:43.785407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:47.315892 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:47.315856    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:47.315892 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:47.315904    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:47.328126 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:47.328089    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:23:47.339172 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:23:47.339146    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:24:08.827616 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:08.827576    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:08.926720 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:08.926686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:13.118771 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:13.118730    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:27.504915 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:27.504837    2580 scope.go:117] "RemoveContainer" containerID="1b2ffa3a6c611683a9056e6e6366c0bd475facc6a958b17d47612a181fd9f3d1"
Jul 18 02:24:39.445079 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:39.445022    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"]
Jul 18 02:24:39.462992 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:39.462949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:40.424892 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.424863    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:40.533490 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.533458    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"]
Jul 18 02:24:40.536865 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.536849    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"
Jul 18 02:24:40.539303 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.539279    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:24:40.545551 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.545527    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"]
Jul 18 02:24:40.596710 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.596660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec22836-1098-4c69-b74d-fe411683f684-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg\" (UID: \"9ec22836-1098-4c69-b74d-fe411683f684\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"
Jul 18 02:24:40.698180 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.698091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec22836-1098-4c69-b74d-fe411683f684-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg\" (UID: \"9ec22836-1098-4c69-b74d-fe411683f684\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"
Jul 18 02:24:40.698711 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.698690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec22836-1098-4c69-b74d-fe411683f684-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg\" (UID: \"9ec22836-1098-4c69-b74d-fe411683f684\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"
Jul 18 02:24:40.848602 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.848571    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"
Jul 18 02:24:40.982520 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:40.982497    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"]
Jul 18 02:24:40.984643 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:24:40.984617    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec22836_1098_4c69_b74d_fe411683f684.slice/crio-35341390c3ba835b45d6cc9aa0e07aeb703953672fece501519cfb30fbd21b04 WatchSource:0}: Error finding container 35341390c3ba835b45d6cc9aa0e07aeb703953672fece501519cfb30fbd21b04: Status 404 returned error can't find the container with id 35341390c3ba835b45d6cc9aa0e07aeb703953672fece501519cfb30fbd21b04
Jul 18 02:24:41.110747 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:41.110711    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:41.209602 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:41.209565    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:41.579720 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:41.579626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg" event={"ID":"9ec22836-1098-4c69-b74d-fe411683f684","Type":"ContainerStarted","Data":"86723d954d9652d223b3d40bbda637f478ba5771e72dfff41f7303b7a81a5b93"}
Jul 18 02:24:41.579720 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:41.579687    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg" event={"ID":"9ec22836-1098-4c69-b74d-fe411683f684","Type":"ContainerStarted","Data":"35341390c3ba835b45d6cc9aa0e07aeb703953672fece501519cfb30fbd21b04"}
Jul 18 02:24:41.594802 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:41.594758    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg" podStartSLOduration=1.5947413259999998 podStartE2EDuration="1.594741326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:24:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:24:41.593683908 +0000 UTC m=+2174.803398129" watchObservedRunningTime="2026-07-18 02:24:41.594741326 +0000 UTC m=+2174.804455547"
Jul 18 02:24:42.585688 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:42.585651    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ec22836-1098-4c69-b74d-fe411683f684" containerID="86723d954d9652d223b3d40bbda637f478ba5771e72dfff41f7303b7a81a5b93" exitCode=0
Jul 18 02:24:42.586118 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:42.585738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg" event={"ID":"9ec22836-1098-4c69-b74d-fe411683f684","Type":"ContainerDied","Data":"86723d954d9652d223b3d40bbda637f478ba5771e72dfff41f7303b7a81a5b93"}
Jul 18 02:24:43.727430 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:43.727403    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"
Jul 18 02:24:43.828263 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:43.828226    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec22836-1098-4c69-b74d-fe411683f684-maas-api-service-ca\") pod \"9ec22836-1098-4c69-b74d-fe411683f684\" (UID: \"9ec22836-1098-4c69-b74d-fe411683f684\") "
Jul 18 02:24:43.828570 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:43.828549    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec22836-1098-4c69-b74d-fe411683f684-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ec22836-1098-4c69-b74d-fe411683f684" (UID: "9ec22836-1098-4c69-b74d-fe411683f684"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:24:43.929768 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:43.929731    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec22836-1098-4c69-b74d-fe411683f684-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:44.599112 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:44.599080    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"
Jul 18 02:24:44.599112 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:44.599084    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg" event={"ID":"9ec22836-1098-4c69-b74d-fe411683f684","Type":"ContainerDied","Data":"35341390c3ba835b45d6cc9aa0e07aeb703953672fece501519cfb30fbd21b04"}
Jul 18 02:24:44.599332 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:44.599140    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35341390c3ba835b45d6cc9aa0e07aeb703953672fece501519cfb30fbd21b04"
Jul 18 02:24:50.884569 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:50.884536    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"]
Jul 18 02:24:50.885132 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:50.884747    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x" podUID="247aaa08-fd9f-4ee6-a7d5-15395a5335ff" containerName="maas-api" containerID="cri-o://8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221" gracePeriod=30
Jul 18 02:24:50.911269 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:50.911226    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:51.168389 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.168348    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x" podUID="247aaa08-fd9f-4ee6-a7d5-15395a5335ff" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.42:8443/health\": dial tcp 10.134.0.42:8443: connect: connection refused"
Jul 18 02:24:51.331015 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.330992    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:24:51.398215 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.398169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-maas-api-tls\") pod \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") "
Jul 18 02:24:51.398402 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.398260    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-openshift-service-ca\") pod \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") "
Jul 18 02:24:51.398402 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.398361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx6sb\" (UniqueName: \"kubernetes.io/projected/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-kube-api-access-bx6sb\") pod \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\" (UID: \"247aaa08-fd9f-4ee6-a7d5-15395a5335ff\") "
Jul 18 02:24:51.398584 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.398562    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "247aaa08-fd9f-4ee6-a7d5-15395a5335ff" (UID: "247aaa08-fd9f-4ee6-a7d5-15395a5335ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:24:51.398659 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.398646    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:51.400360 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.400326    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "247aaa08-fd9f-4ee6-a7d5-15395a5335ff" (UID: "247aaa08-fd9f-4ee6-a7d5-15395a5335ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:24:51.400461 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.400443    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-kube-api-access-bx6sb" (OuterVolumeSpecName: "kube-api-access-bx6sb") pod "247aaa08-fd9f-4ee6-a7d5-15395a5335ff" (UID: "247aaa08-fd9f-4ee6-a7d5-15395a5335ff"). InnerVolumeSpecName "kube-api-access-bx6sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:24:51.499626 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.499524    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx6sb\" (UniqueName: \"kubernetes.io/projected/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-kube-api-access-bx6sb\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:51.499626 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.499563    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/247aaa08-fd9f-4ee6-a7d5-15395a5335ff-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:51.633068 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.633014    2580 generic.go:356] "Generic (PLEG): container finished" podID="247aaa08-fd9f-4ee6-a7d5-15395a5335ff" containerID="8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221" exitCode=0
Jul 18 02:24:51.633233 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.633088    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"
Jul 18 02:24:51.633233 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.633100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x" event={"ID":"247aaa08-fd9f-4ee6-a7d5-15395a5335ff","Type":"ContainerDied","Data":"8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221"}
Jul 18 02:24:51.633233 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.633155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x" event={"ID":"247aaa08-fd9f-4ee6-a7d5-15395a5335ff","Type":"ContainerDied","Data":"ee2437cd6c26aa654bc129c212cb9f1b3883dfc73cd35cd3a6883235ef3c3865"}
Jul 18 02:24:51.633233 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.633176    2580 scope.go:117] "RemoveContainer" containerID="8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221"
Jul 18 02:24:51.642649 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.642630    2580 scope.go:117] "RemoveContainer" containerID="8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221"
Jul 18 02:24:51.642993 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:24:51.642973    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221\": container with ID starting with 8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221 not found: ID does not exist" containerID="8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221"
Jul 18 02:24:51.643112 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.643002    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221"} err="failed to get container status \"8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221\": rpc error: code = NotFound desc = could not find container \"8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221\": container with ID starting with 8438dfd6fd87ab002c70abb42adf2ee90ceac28d0c490a91524bd4c7199d7221 not found: ID does not exist"
Jul 18 02:24:51.654183 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.654160    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"]
Jul 18 02:24:51.657607 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:51.657587    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8b6a54-5d4dbd7786-jp77x"]
Jul 18 02:24:53.390497 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:53.390462    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="247aaa08-fd9f-4ee6-a7d5-15395a5335ff" path="/var/lib/kubelet/pods/247aaa08-fd9f-4ee6-a7d5-15395a5335ff/volumes"
Jul 18 02:24:54.467101 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.467033    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl" podUID="0abb5927-eb48-4efd-8af0-497a44e4b2d7" containerName="main" containerID="cri-o://69713760dd2143e6310ab64ab45fe9c35f29651d37c1e0b961919902ce8decea" gracePeriod=45
Jul 18 02:24:54.649190 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.649159    2580 generic.go:356] "Generic (PLEG): container finished" podID="0abb5927-eb48-4efd-8af0-497a44e4b2d7" containerID="69713760dd2143e6310ab64ab45fe9c35f29651d37c1e0b961919902ce8decea" exitCode=0
Jul 18 02:24:54.649359 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.649238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl" event={"ID":"0abb5927-eb48-4efd-8af0-497a44e4b2d7","Type":"ContainerDied","Data":"69713760dd2143e6310ab64ab45fe9c35f29651d37c1e0b961919902ce8decea"}
Jul 18 02:24:54.719704 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.719646    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:24:54.733148 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.733116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:54.831157 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831127    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-model-cache\") pod \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") "
Jul 18 02:24:54.831338 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831180    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-dshm\") pod \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") "
Jul 18 02:24:54.831338 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tls-certs\") pod \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") "
Jul 18 02:24:54.831338 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831238    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tmp-dir\") pod \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") "
Jul 18 02:24:54.831338 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831259    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-home\") pod \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") "
Jul 18 02:24:54.831338 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831292    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f4r7\" (UniqueName: \"kubernetes.io/projected/0abb5927-eb48-4efd-8af0-497a44e4b2d7-kube-api-access-6f4r7\") pod \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\" (UID: \"0abb5927-eb48-4efd-8af0-497a44e4b2d7\") "
Jul 18 02:24:54.831608 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831438    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "0abb5927-eb48-4efd-8af0-497a44e4b2d7" (UID: "0abb5927-eb48-4efd-8af0-497a44e4b2d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:24:54.831608 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831534    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-home" (OuterVolumeSpecName: "home") pod "0abb5927-eb48-4efd-8af0-497a44e4b2d7" (UID: "0abb5927-eb48-4efd-8af0-497a44e4b2d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:24:54.831608 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831549    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0abb5927-eb48-4efd-8af0-497a44e4b2d7" (UID: "0abb5927-eb48-4efd-8af0-497a44e4b2d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:24:54.831752 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831672    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-model-cache\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:54.831752 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831691    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tmp-dir\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:54.831752 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.831704    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-home\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:54.833635 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.833613    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0abb5927-eb48-4efd-8af0-497a44e4b2d7" (UID: "0abb5927-eb48-4efd-8af0-497a44e4b2d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:24:54.833760 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.833689    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-dshm" (OuterVolumeSpecName: "dshm") pod "0abb5927-eb48-4efd-8af0-497a44e4b2d7" (UID: "0abb5927-eb48-4efd-8af0-497a44e4b2d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:24:54.833760 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.833691    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0abb5927-eb48-4efd-8af0-497a44e4b2d7-kube-api-access-6f4r7" (OuterVolumeSpecName: "kube-api-access-6f4r7") pod "0abb5927-eb48-4efd-8af0-497a44e4b2d7" (UID: "0abb5927-eb48-4efd-8af0-497a44e4b2d7"). InnerVolumeSpecName "kube-api-access-6f4r7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:24:54.938479 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.933579    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0abb5927-eb48-4efd-8af0-497a44e4b2d7-dshm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:54.938479 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.933617    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0abb5927-eb48-4efd-8af0-497a44e4b2d7-tls-certs\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:54.938479 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:54.933641    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f4r7\" (UniqueName: \"kubernetes.io/projected/0abb5927-eb48-4efd-8af0-497a44e4b2d7-kube-api-access-6f4r7\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:24:55.597449 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:55.597412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:55.607943 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:55.607913    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"]
Jul 18 02:24:55.612770 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:55.612747    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8b6a54-cwnjg"]
Jul 18 02:24:55.657264 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:55.657187    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl" event={"ID":"0abb5927-eb48-4efd-8af0-497a44e4b2d7","Type":"ContainerDied","Data":"264a10114b05f9abf710018a06b319b21a6e2223d670e51a806612b7332e3792"}
Jul 18 02:24:55.657264 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:55.657206    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"
Jul 18 02:24:55.657264 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:55.657239    2580 scope.go:117] "RemoveContainer" containerID="69713760dd2143e6310ab64ab45fe9c35f29651d37c1e0b961919902ce8decea"
Jul 18 02:24:55.679231 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:55.679205    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"]
Jul 18 02:24:55.680996 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:55.680973    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8b6a54/test-model-8b6a54-kserve-77bb8956d9-k7pzl"]
Jul 18 02:24:55.973778 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:55.973748    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:57.393901 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:57.393865    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0abb5927-eb48-4efd-8af0-497a44e4b2d7" path="/var/lib/kubelet/pods/0abb5927-eb48-4efd-8af0-497a44e4b2d7/volumes"
Jul 18 02:24:57.394306 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:24:57.394215    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ec22836-1098-4c69-b74d-fe411683f684" path="/var/lib/kubelet/pods/9ec22836-1098-4c69-b74d-fe411683f684/volumes"
Jul 18 02:25:01.798404 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:01.798373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:01.866553 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:01.866515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:02.065247 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.065161    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:02.278596 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.278558    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:02.391313 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391233    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"]
Jul 18 02:25:02.391654 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391640    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec22836-1098-4c69-b74d-fe411683f684" containerName="revoke-keys"
Jul 18 02:25:02.391707 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391660    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec22836-1098-4c69-b74d-fe411683f684" containerName="revoke-keys"
Jul 18 02:25:02.391707 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391674    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0abb5927-eb48-4efd-8af0-497a44e4b2d7" containerName="main"
Jul 18 02:25:02.391707 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391680    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0abb5927-eb48-4efd-8af0-497a44e4b2d7" containerName="main"
Jul 18 02:25:02.391707 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391704    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247aaa08-fd9f-4ee6-a7d5-15395a5335ff" containerName="maas-api"
Jul 18 02:25:02.391707 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391709    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="247aaa08-fd9f-4ee6-a7d5-15395a5335ff" containerName="maas-api"
Jul 18 02:25:02.391882 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391768    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec22836-1098-4c69-b74d-fe411683f684" containerName="revoke-keys"
Jul 18 02:25:02.391882 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391780    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="247aaa08-fd9f-4ee6-a7d5-15395a5335ff" containerName="maas-api"
Jul 18 02:25:02.391882 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.391786    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0abb5927-eb48-4efd-8af0-497a44e4b2d7" containerName="main"
Jul 18 02:25:02.394915 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.394898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"
Jul 18 02:25:02.397773 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.397742    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:25:02.403257 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.403237    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"]
Jul 18 02:25:02.500641 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.500609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a7083a3-a74a-4ce4-a92f-02902841a54d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9\" (UID: \"4a7083a3-a74a-4ce4-a92f-02902841a54d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"
Jul 18 02:25:02.601582 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.601549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a7083a3-a74a-4ce4-a92f-02902841a54d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9\" (UID: \"4a7083a3-a74a-4ce4-a92f-02902841a54d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"
Jul 18 02:25:02.602183 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.602166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a7083a3-a74a-4ce4-a92f-02902841a54d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9\" (UID: \"4a7083a3-a74a-4ce4-a92f-02902841a54d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"
Jul 18 02:25:02.705601 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.705575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"
Jul 18 02:25:02.839172 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.839150    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"]
Jul 18 02:25:02.841004 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:25:02.840977    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7083a3_a74a_4ce4_a92f_02902841a54d.slice/crio-7e0cd98c8f8a118113b638843d6fbbf785c4ed81ed281b190ee770eb04e3ab01 WatchSource:0}: Error finding container 7e0cd98c8f8a118113b638843d6fbbf785c4ed81ed281b190ee770eb04e3ab01: Status 404 returned error can't find the container with id 7e0cd98c8f8a118113b638843d6fbbf785c4ed81ed281b190ee770eb04e3ab01
Jul 18 02:25:02.907800 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:02.907768    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:03.008084 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:03.007986    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:03.527022 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:25:03.526993    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7083a3_a74a_4ce4_a92f_02902841a54d.slice/crio-conmon-29cb326abf0dc682eff4833585ea456a50a6db6ec7c4b51030a89883834617b7.scope\": RecentStats: unable to find data in memory cache]"
Jul 18 02:25:03.696920 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:03.696887    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a7083a3-a74a-4ce4-a92f-02902841a54d" containerID="29cb326abf0dc682eff4833585ea456a50a6db6ec7c4b51030a89883834617b7" exitCode=0
Jul 18 02:25:03.697112 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:03.696964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9" event={"ID":"4a7083a3-a74a-4ce4-a92f-02902841a54d","Type":"ContainerDied","Data":"29cb326abf0dc682eff4833585ea456a50a6db6ec7c4b51030a89883834617b7"}
Jul 18 02:25:03.697112 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:03.697020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9" event={"ID":"4a7083a3-a74a-4ce4-a92f-02902841a54d","Type":"ContainerStarted","Data":"7e0cd98c8f8a118113b638843d6fbbf785c4ed81ed281b190ee770eb04e3ab01"}
Jul 18 02:25:04.841963 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:04.841938    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"
Jul 18 02:25:05.024981 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:05.024903    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a7083a3-a74a-4ce4-a92f-02902841a54d-maas-api-service-ca\") pod \"4a7083a3-a74a-4ce4-a92f-02902841a54d\" (UID: \"4a7083a3-a74a-4ce4-a92f-02902841a54d\") "
Jul 18 02:25:05.025269 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:05.025248    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a7083a3-a74a-4ce4-a92f-02902841a54d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a7083a3-a74a-4ce4-a92f-02902841a54d" (UID: "4a7083a3-a74a-4ce4-a92f-02902841a54d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:25:05.125882 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:05.125851    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a7083a3-a74a-4ce4-a92f-02902841a54d-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:25:05.707648 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:05.707622    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"
Jul 18 02:25:05.707648 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:05.707633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9" event={"ID":"4a7083a3-a74a-4ce4-a92f-02902841a54d","Type":"ContainerDied","Data":"7e0cd98c8f8a118113b638843d6fbbf785c4ed81ed281b190ee770eb04e3ab01"}
Jul 18 02:25:05.707843 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:05.707678    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e0cd98c8f8a118113b638843d6fbbf785c4ed81ed281b190ee770eb04e3ab01"
Jul 18 02:25:12.968851 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:12.968818    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"]
Jul 18 02:25:12.969423 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:12.969031    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z" podUID="992e1281-2bc3-4f17-9676-72cd6bddc526" containerName="maas-api" containerID="cri-o://f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231" gracePeriod=30
Jul 18 02:25:13.008846 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.008818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:13.221690 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.221623    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:25:13.297029 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.297002    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbhnm\" (UniqueName: \"kubernetes.io/projected/992e1281-2bc3-4f17-9676-72cd6bddc526-kube-api-access-dbhnm\") pod \"992e1281-2bc3-4f17-9676-72cd6bddc526\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") "
Jul 18 02:25:13.297223 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.297097    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992e1281-2bc3-4f17-9676-72cd6bddc526-maas-api-tls\") pod \"992e1281-2bc3-4f17-9676-72cd6bddc526\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") "
Jul 18 02:25:13.297223 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.297170    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992e1281-2bc3-4f17-9676-72cd6bddc526-openshift-service-ca\") pod \"992e1281-2bc3-4f17-9676-72cd6bddc526\" (UID: \"992e1281-2bc3-4f17-9676-72cd6bddc526\") "
Jul 18 02:25:13.297520 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.297496    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/992e1281-2bc3-4f17-9676-72cd6bddc526-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "992e1281-2bc3-4f17-9676-72cd6bddc526" (UID: "992e1281-2bc3-4f17-9676-72cd6bddc526"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:25:13.299225 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.299201    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/992e1281-2bc3-4f17-9676-72cd6bddc526-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "992e1281-2bc3-4f17-9676-72cd6bddc526" (UID: "992e1281-2bc3-4f17-9676-72cd6bddc526"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:25:13.299225 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.299202    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/992e1281-2bc3-4f17-9676-72cd6bddc526-kube-api-access-dbhnm" (OuterVolumeSpecName: "kube-api-access-dbhnm") pod "992e1281-2bc3-4f17-9676-72cd6bddc526" (UID: "992e1281-2bc3-4f17-9676-72cd6bddc526"). InnerVolumeSpecName "kube-api-access-dbhnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:25:13.397736 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.397709    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992e1281-2bc3-4f17-9676-72cd6bddc526-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:25:13.397736 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.397737    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbhnm\" (UniqueName: \"kubernetes.io/projected/992e1281-2bc3-4f17-9676-72cd6bddc526-kube-api-access-dbhnm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:25:13.397920 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.397747    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992e1281-2bc3-4f17-9676-72cd6bddc526-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:25:13.746807 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.746764    2580 generic.go:356] "Generic (PLEG): container finished" podID="992e1281-2bc3-4f17-9676-72cd6bddc526" containerID="f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231" exitCode=0
Jul 18 02:25:13.746987 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.746829    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"
Jul 18 02:25:13.746987 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.746843    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z" event={"ID":"992e1281-2bc3-4f17-9676-72cd6bddc526","Type":"ContainerDied","Data":"f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231"}
Jul 18 02:25:13.746987 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.746919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z" event={"ID":"992e1281-2bc3-4f17-9676-72cd6bddc526","Type":"ContainerDied","Data":"bbd7a635bf3760aa624bf93a5a1a24db04de65496f9da5a9488f173ee8541e78"}
Jul 18 02:25:13.746987 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.746942    2580 scope.go:117] "RemoveContainer" containerID="f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231"
Jul 18 02:25:13.755487 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.755457    2580 scope.go:117] "RemoveContainer" containerID="f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231"
Jul 18 02:25:13.755792 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:25:13.755766    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231\": container with ID starting with f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231 not found: ID does not exist" containerID="f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231"
Jul 18 02:25:13.755880 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.755803    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231"} err="failed to get container status \"f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231\": rpc error: code = NotFound desc = could not find container \"f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231\": container with ID starting with f36ed7fc17133740a3b5a3ac04650357db55fa89561856c555feb89d73dbf231 not found: ID does not exist"
Jul 18 02:25:13.764243 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.764194    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"]
Jul 18 02:25:13.766430 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:13.766410    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f21b76-67d66b67f7-6rk6z"]
Jul 18 02:25:15.391133 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:15.391098    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="992e1281-2bc3-4f17-9676-72cd6bddc526" path="/var/lib/kubelet/pods/992e1281-2bc3-4f17-9676-72cd6bddc526/volumes"
Jul 18 02:25:17.485279 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:17.485233    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"]
Jul 18 02:25:17.487609 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:17.487588    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f21b76-tjwt9"]
Jul 18 02:25:17.492975 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:17.492950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:17.848538 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:17.848456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:19.390573 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:19.390542    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a7083a3-a74a-4ce4-a92f-02902841a54d" path="/var/lib/kubelet/pods/4a7083a3-a74a-4ce4-a92f-02902841a54d/volumes"
Jul 18 02:25:23.623459 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:23.623422    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:23.693106 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:23.693037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:23.900745 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:23.900650    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:24.104908 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:24.104872    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:24.437766 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:24.437735    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:24.733611 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:24.733515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:24.870791 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:24.870755    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:27.915485 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:27.915400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:28.016097 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.016012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:28.022076 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.022027    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"]
Jul 18 02:25:28.022410 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.022398    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a7083a3-a74a-4ce4-a92f-02902841a54d" containerName="revoke-keys"
Jul 18 02:25:28.022455 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.022412    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a7083a3-a74a-4ce4-a92f-02902841a54d" containerName="revoke-keys"
Jul 18 02:25:28.022455 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.022420    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="992e1281-2bc3-4f17-9676-72cd6bddc526" containerName="maas-api"
Jul 18 02:25:28.022455 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.022426    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="992e1281-2bc3-4f17-9676-72cd6bddc526" containerName="maas-api"
Jul 18 02:25:28.022550 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.022487    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="992e1281-2bc3-4f17-9676-72cd6bddc526" containerName="maas-api"
Jul 18 02:25:28.022550 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.022496    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a7083a3-a74a-4ce4-a92f-02902841a54d" containerName="revoke-keys"
Jul 18 02:25:28.026915 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.026897    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.029948 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.029928    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-58b546-serving-cert\""
Jul 18 02:25:28.030079 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.029958    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:25:28.030079 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.030009    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:25:28.031204 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.031168    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:25:28.036150 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.036130    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"]
Jul 18 02:25:28.131897 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.131862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40af7e37-b28e-44d1-bd40-b78d39579e88-openshift-service-ca\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.132109 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.131927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fctps\" (UniqueName: \"kubernetes.io/projected/40af7e37-b28e-44d1-bd40-b78d39579e88-kube-api-access-fctps\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.132109 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.131976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40af7e37-b28e-44d1-bd40-b78d39579e88-maas-api-tls\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.136223 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.136196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:28.232590 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.232543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40af7e37-b28e-44d1-bd40-b78d39579e88-openshift-service-ca\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.232789 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.232612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fctps\" (UniqueName: \"kubernetes.io/projected/40af7e37-b28e-44d1-bd40-b78d39579e88-kube-api-access-fctps\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.232789 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.232641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40af7e37-b28e-44d1-bd40-b78d39579e88-maas-api-tls\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.232789 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:25:28.232729    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-serving-cert: secret "maas-api-e2e-api-a-58b546-serving-cert" not found
Jul 18 02:25:28.232789 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:25:28.232790    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40af7e37-b28e-44d1-bd40-b78d39579e88-maas-api-tls podName:40af7e37-b28e-44d1-bd40-b78d39579e88 nodeName:}" failed. No retries permitted until 2026-07-18 02:25:28.732773897 +0000 UTC m=+2221.942488095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/40af7e37-b28e-44d1-bd40-b78d39579e88-maas-api-tls") pod "maas-api-e2e-api-a-58b546-57b994675-wsnkq" (UID: "40af7e37-b28e-44d1-bd40-b78d39579e88") : secret "maas-api-e2e-api-a-58b546-serving-cert" not found
Jul 18 02:25:28.233322 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.233300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40af7e37-b28e-44d1-bd40-b78d39579e88-openshift-service-ca\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.254453 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.254421    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fctps\" (UniqueName: \"kubernetes.io/projected/40af7e37-b28e-44d1-bd40-b78d39579e88-kube-api-access-fctps\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.506297 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.506216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:28.667548 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.667516    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:28.737654 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.737619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40af7e37-b28e-44d1-bd40-b78d39579e88-maas-api-tls\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.739968 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.739942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40af7e37-b28e-44d1-bd40-b78d39579e88-maas-api-tls\") pod \"maas-api-e2e-api-a-58b546-57b994675-wsnkq\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:28.790294 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.790222    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:28.937857 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:28.937805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:29.071064 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:29.071007    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"]
Jul 18 02:25:29.073685 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:25:29.073651    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40af7e37_b28e_44d1_bd40_b78d39579e88.slice/crio-3547b6a2b11a75d7cac71ff5c6350a6f08856111bfa3a811be133cc3ade889c3 WatchSource:0}: Error finding container 3547b6a2b11a75d7cac71ff5c6350a6f08856111bfa3a811be133cc3ade889c3: Status 404 returned error can't find the container with id 3547b6a2b11a75d7cac71ff5c6350a6f08856111bfa3a811be133cc3ade889c3
Jul 18 02:25:29.827137 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:29.827026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq" event={"ID":"40af7e37-b28e-44d1-bd40-b78d39579e88","Type":"ContainerStarted","Data":"a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910"}
Jul 18 02:25:29.827137 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:29.827114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq" event={"ID":"40af7e37-b28e-44d1-bd40-b78d39579e88","Type":"ContainerStarted","Data":"3547b6a2b11a75d7cac71ff5c6350a6f08856111bfa3a811be133cc3ade889c3"}
Jul 18 02:25:29.827331 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:29.827217    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:29.844960 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:29.844914    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq" podStartSLOduration=1.379217568 podStartE2EDuration="1.8449001s" totalImagesPullingTime="465.682532ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:29.843333985 +0000 UTC m=+2223.053048203" watchObservedRunningTime="2026-07-18 02:25:29.8449001 +0000 UTC m=+2223.054614320"
Jul 18 02:25:32.404888 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:32.404848    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:32.838258 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:32.838216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:33.137508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:33.137408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:33.365892 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:33.365860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:35.744200 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.744160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:35.839066 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.839017    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:25:35.868144 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.868116    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"]
Jul 18 02:25:35.871629 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.871607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:35.874629 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.874611    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0e9c76-serving-cert\""
Jul 18 02:25:35.890097 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.890077    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"]
Jul 18 02:25:35.922074 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.922029    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:35.999695 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.999608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hdwn\" (UniqueName: \"kubernetes.io/projected/c9be942f-b493-4895-bc17-0e4b925d5570-kube-api-access-7hdwn\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:35.999695 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.999646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9be942f-b493-4895-bc17-0e4b925d5570-maas-api-tls\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:35.999695 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:35.999666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9be942f-b493-4895-bc17-0e4b925d5570-openshift-service-ca\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:36.023089 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.023036    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:36.100391 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.100349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hdwn\" (UniqueName: \"kubernetes.io/projected/c9be942f-b493-4895-bc17-0e4b925d5570-kube-api-access-7hdwn\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:36.100596 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.100410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9be942f-b493-4895-bc17-0e4b925d5570-maas-api-tls\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:36.100596 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.100441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9be942f-b493-4895-bc17-0e4b925d5570-openshift-service-ca\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:36.101236 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:25:36.101216    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-serving-cert: secret "maas-api-e2e-api-b-0e9c76-serving-cert" not found
Jul 18 02:25:36.101458 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:25:36.101433    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9be942f-b493-4895-bc17-0e4b925d5570-maas-api-tls podName:c9be942f-b493-4895-bc17-0e4b925d5570 nodeName:}" failed. No retries permitted until 2026-07-18 02:25:36.601411671 +0000 UTC m=+2229.811125875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9be942f-b493-4895-bc17-0e4b925d5570-maas-api-tls") pod "maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8" (UID: "c9be942f-b493-4895-bc17-0e4b925d5570") : secret "maas-api-e2e-api-b-0e9c76-serving-cert" not found
Jul 18 02:25:36.101562 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.101326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9be942f-b493-4895-bc17-0e4b925d5570-openshift-service-ca\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:36.111520 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.111498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hdwn\" (UniqueName: \"kubernetes.io/projected/c9be942f-b493-4895-bc17-0e4b925d5570-kube-api-access-7hdwn\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:36.228599 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.228563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:36.472287 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.472253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:36.576187 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.576160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:36.604747 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.604714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9be942f-b493-4895-bc17-0e4b925d5570-maas-api-tls\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:36.607167 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.607149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9be942f-b493-4895-bc17-0e4b925d5570-maas-api-tls\") pod \"maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:36.782245 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.782162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:36.913808 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.913782    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"]
Jul 18 02:25:36.916239 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:25:36.916210    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9be942f_b493_4895_bc17_0e4b925d5570.slice/crio-e0db8f2b78626584246e26c07da0cff756ff4929da76e35d619b28035e1237e7 WatchSource:0}: Error finding container e0db8f2b78626584246e26c07da0cff756ff4929da76e35d619b28035e1237e7: Status 404 returned error can't find the container with id e0db8f2b78626584246e26c07da0cff756ff4929da76e35d619b28035e1237e7
Jul 18 02:25:36.917756 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:36.917733    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:25:37.866832 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:37.866788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8" event={"ID":"c9be942f-b493-4895-bc17-0e4b925d5570","Type":"ContainerStarted","Data":"94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2"}
Jul 18 02:25:37.867287 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:37.866864    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:37.867287 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:37.866883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8" event={"ID":"c9be942f-b493-4895-bc17-0e4b925d5570","Type":"ContainerStarted","Data":"e0db8f2b78626584246e26c07da0cff756ff4929da76e35d619b28035e1237e7"}
Jul 18 02:25:37.885767 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:37.885719    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8" podStartSLOduration=2.522459846 podStartE2EDuration="2.885705015s" totalImagesPullingTime="363.245169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:37.884120789 +0000 UTC m=+2231.093835011" watchObservedRunningTime="2026-07-18 02:25:37.885705015 +0000 UTC m=+2231.095419236"
Jul 18 02:25:40.171291 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:40.171260    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:43.877851 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:43.877817    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:25:44.068872 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:44.068837    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:48.154874 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.154786    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"]
Jul 18 02:25:48.158325 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.158303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"
Jul 18 02:25:48.161130 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.161112    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:25:48.165493 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.165474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"]
Jul 18 02:25:48.309457 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.309417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/374e1eb3-3d06-4638-ab14-aa7a202d15b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-58b546-rhd8f\" (UID: \"374e1eb3-3d06-4638-ab14-aa7a202d15b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"
Jul 18 02:25:48.410034 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.409960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/374e1eb3-3d06-4638-ab14-aa7a202d15b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-58b546-rhd8f\" (UID: \"374e1eb3-3d06-4638-ab14-aa7a202d15b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"
Jul 18 02:25:48.410573 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.410555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/374e1eb3-3d06-4638-ab14-aa7a202d15b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-58b546-rhd8f\" (UID: \"374e1eb3-3d06-4638-ab14-aa7a202d15b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"
Jul 18 02:25:48.470919 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.470888    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"
Jul 18 02:25:48.602679 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.602651    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"]
Jul 18 02:25:48.604760 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:25:48.604734    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod374e1eb3_3d06_4638_ab14_aa7a202d15b6.slice/crio-617d1281f8de4c3b0dd75b453079e0348db6362c0ac99c68300f9a7667fdb8b3 WatchSource:0}: Error finding container 617d1281f8de4c3b0dd75b453079e0348db6362c0ac99c68300f9a7667fdb8b3: Status 404 returned error can't find the container with id 617d1281f8de4c3b0dd75b453079e0348db6362c0ac99c68300f9a7667fdb8b3
Jul 18 02:25:48.800771 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.800730    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:48.898653 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.898621    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:48.918520 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:48.918482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f" event={"ID":"374e1eb3-3d06-4638-ab14-aa7a202d15b6","Type":"ContainerStarted","Data":"617d1281f8de4c3b0dd75b453079e0348db6362c0ac99c68300f9a7667fdb8b3"}
Jul 18 02:25:49.925604 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:49.925546    2580 generic.go:356] "Generic (PLEG): container finished" podID="374e1eb3-3d06-4638-ab14-aa7a202d15b6" containerID="3cbfc3f9325d666a62d4ffceee0f528a588f5f55a8475c3a5e1de66ff5a6f6ff" exitCode=0
Jul 18 02:25:49.925604 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:49.925589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f" event={"ID":"374e1eb3-3d06-4638-ab14-aa7a202d15b6","Type":"ContainerDied","Data":"3cbfc3f9325d666a62d4ffceee0f528a588f5f55a8475c3a5e1de66ff5a6f6ff"}
Jul 18 02:25:51.058133 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:51.058104    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"
Jul 18 02:25:51.234364 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:51.234284    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/374e1eb3-3d06-4638-ab14-aa7a202d15b6-maas-api-service-ca\") pod \"374e1eb3-3d06-4638-ab14-aa7a202d15b6\" (UID: \"374e1eb3-3d06-4638-ab14-aa7a202d15b6\") "
Jul 18 02:25:51.234614 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:51.234592    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374e1eb3-3d06-4638-ab14-aa7a202d15b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "374e1eb3-3d06-4638-ab14-aa7a202d15b6" (UID: "374e1eb3-3d06-4638-ab14-aa7a202d15b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:25:51.334820 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:51.334792    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/374e1eb3-3d06-4638-ab14-aa7a202d15b6-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:25:51.936309 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:51.936280    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"
Jul 18 02:25:51.936309 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:51.936293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f" event={"ID":"374e1eb3-3d06-4638-ab14-aa7a202d15b6","Type":"ContainerDied","Data":"617d1281f8de4c3b0dd75b453079e0348db6362c0ac99c68300f9a7667fdb8b3"}
Jul 18 02:25:51.936529 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:51.936340    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="617d1281f8de4c3b0dd75b453079e0348db6362c0ac99c68300f9a7667fdb8b3"
Jul 18 02:25:57.331560 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:57.331527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:57.430242 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:57.430202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:58.677207 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:58.677172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:58.779083 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:25:58.779013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:03.906021 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:03.905990    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"]
Jul 18 02:26:03.906534 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:03.906229    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq" podUID="40af7e37-b28e-44d1-bd40-b78d39579e88" containerName="maas-api" containerID="cri-o://a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910" gracePeriod=30
Jul 18 02:26:03.932034 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:03.932005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:04.154288 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.154262    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:26:04.257199 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.257170    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40af7e37-b28e-44d1-bd40-b78d39579e88-maas-api-tls\") pod \"40af7e37-b28e-44d1-bd40-b78d39579e88\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") "
Jul 18 02:26:04.257403 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.257219    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40af7e37-b28e-44d1-bd40-b78d39579e88-openshift-service-ca\") pod \"40af7e37-b28e-44d1-bd40-b78d39579e88\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") "
Jul 18 02:26:04.257403 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.257275    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fctps\" (UniqueName: \"kubernetes.io/projected/40af7e37-b28e-44d1-bd40-b78d39579e88-kube-api-access-fctps\") pod \"40af7e37-b28e-44d1-bd40-b78d39579e88\" (UID: \"40af7e37-b28e-44d1-bd40-b78d39579e88\") "
Jul 18 02:26:04.257654 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.257629    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40af7e37-b28e-44d1-bd40-b78d39579e88-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40af7e37-b28e-44d1-bd40-b78d39579e88" (UID: "40af7e37-b28e-44d1-bd40-b78d39579e88"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:26:04.259435 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.259407    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40af7e37-b28e-44d1-bd40-b78d39579e88-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40af7e37-b28e-44d1-bd40-b78d39579e88" (UID: "40af7e37-b28e-44d1-bd40-b78d39579e88"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:26:04.259521 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.259403    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40af7e37-b28e-44d1-bd40-b78d39579e88-kube-api-access-fctps" (OuterVolumeSpecName: "kube-api-access-fctps") pod "40af7e37-b28e-44d1-bd40-b78d39579e88" (UID: "40af7e37-b28e-44d1-bd40-b78d39579e88"). InnerVolumeSpecName "kube-api-access-fctps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:26:04.358053 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.358017    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40af7e37-b28e-44d1-bd40-b78d39579e88-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:26:04.358191 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.358076    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40af7e37-b28e-44d1-bd40-b78d39579e88-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:26:04.358191 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.358087    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fctps\" (UniqueName: \"kubernetes.io/projected/40af7e37-b28e-44d1-bd40-b78d39579e88-kube-api-access-fctps\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:26:04.998390 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.998353    2580 generic.go:356] "Generic (PLEG): container finished" podID="40af7e37-b28e-44d1-bd40-b78d39579e88" containerID="a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910" exitCode=0
Jul 18 02:26:04.998798 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.998395    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"
Jul 18 02:26:04.998798 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.998441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq" event={"ID":"40af7e37-b28e-44d1-bd40-b78d39579e88","Type":"ContainerDied","Data":"a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910"}
Jul 18 02:26:04.998798 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.998492    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq" event={"ID":"40af7e37-b28e-44d1-bd40-b78d39579e88","Type":"ContainerDied","Data":"3547b6a2b11a75d7cac71ff5c6350a6f08856111bfa3a811be133cc3ade889c3"}
Jul 18 02:26:04.998798 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:04.998518    2580 scope.go:117] "RemoveContainer" containerID="a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910"
Jul 18 02:26:05.007798 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:05.007781    2580 scope.go:117] "RemoveContainer" containerID="a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910"
Jul 18 02:26:05.008035 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:26:05.008017    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910\": container with ID starting with a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910 not found: ID does not exist" containerID="a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910"
Jul 18 02:26:05.008114 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:05.008061    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910"} err="failed to get container status \"a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910\": rpc error: code = NotFound desc = could not find container \"a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910\": container with ID starting with a8c55c1a302c0e6a09a75f20e9e8cf584dddc64249ed90944331673cf2b3e910 not found: ID does not exist"
Jul 18 02:26:05.020960 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:05.020935    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"]
Jul 18 02:26:05.024251 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:05.024219    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-58b546-57b994675-wsnkq"]
Jul 18 02:26:05.390729 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:05.390632    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40af7e37-b28e-44d1-bd40-b78d39579e88" path="/var/lib/kubelet/pods/40af7e37-b28e-44d1-bd40-b78d39579e88/volumes"
Jul 18 02:26:08.221177 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:08.221144    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"]
Jul 18 02:26:08.223163 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:08.223140    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-58b546-rhd8f"]
Jul 18 02:26:08.232315 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:08.227610    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:08.601870 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:08.601783    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:09.391015 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:09.390978    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="374e1eb3-3d06-4638-ab14-aa7a202d15b6" path="/var/lib/kubelet/pods/374e1eb3-3d06-4638-ab14-aa7a202d15b6/volumes"
Jul 18 02:26:14.576680 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:14.576642    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:14.636913 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:14.636884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:14.829832 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:14.829740    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:15.057031 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.057003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:15.172597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.172566    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"]
Jul 18 02:26:15.172948 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.172936    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40af7e37-b28e-44d1-bd40-b78d39579e88" containerName="maas-api"
Jul 18 02:26:15.172997 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.172950    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="40af7e37-b28e-44d1-bd40-b78d39579e88" containerName="maas-api"
Jul 18 02:26:15.172997 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.172957    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374e1eb3-3d06-4638-ab14-aa7a202d15b6" containerName="revoke-keys"
Jul 18 02:26:15.172997 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.172963    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="374e1eb3-3d06-4638-ab14-aa7a202d15b6" containerName="revoke-keys"
Jul 18 02:26:15.173124 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.173018    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="374e1eb3-3d06-4638-ab14-aa7a202d15b6" containerName="revoke-keys"
Jul 18 02:26:15.173124 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.173031    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="40af7e37-b28e-44d1-bd40-b78d39579e88" containerName="maas-api"
Jul 18 02:26:15.178534 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.178509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"
Jul 18 02:26:15.182604 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.182573    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:26:15.184927 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.184903    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"]
Jul 18 02:26:15.256698 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.256658    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e388945-71ba-4ed3-a7e5-e4065c1b038f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp\" (UID: \"8e388945-71ba-4ed3-a7e5-e4065c1b038f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"
Jul 18 02:26:15.357463 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.357419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e388945-71ba-4ed3-a7e5-e4065c1b038f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp\" (UID: \"8e388945-71ba-4ed3-a7e5-e4065c1b038f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"
Jul 18 02:26:15.358031 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.358012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e388945-71ba-4ed3-a7e5-e4065c1b038f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp\" (UID: \"8e388945-71ba-4ed3-a7e5-e4065c1b038f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"
Jul 18 02:26:15.492390 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.492313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"
Jul 18 02:26:15.621269 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.621242    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"]
Jul 18 02:26:15.623135 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:26:15.623103    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e388945_71ba_4ed3_a7e5_e4065c1b038f.slice/crio-cecdadad1f0fb4e1953840b4046ffd0df5cbbdc1aba7f1cae8eb2003e660ec69 WatchSource:0}: Error finding container cecdadad1f0fb4e1953840b4046ffd0df5cbbdc1aba7f1cae8eb2003e660ec69: Status 404 returned error can't find the container with id cecdadad1f0fb4e1953840b4046ffd0df5cbbdc1aba7f1cae8eb2003e660ec69
Jul 18 02:26:15.700106 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.700069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:15.800268 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:15.800170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:16.052773 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:16.052690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp" event={"ID":"8e388945-71ba-4ed3-a7e5-e4065c1b038f","Type":"ContainerStarted","Data":"2cfb7c8e1b9d4b4b2f34bdb25dbb3d4a87839bea3d42adc62665c437a3c17a39"}
Jul 18 02:26:16.052912 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:16.052773    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp" event={"ID":"8e388945-71ba-4ed3-a7e5-e4065c1b038f","Type":"ContainerStarted","Data":"cecdadad1f0fb4e1953840b4046ffd0df5cbbdc1aba7f1cae8eb2003e660ec69"}
Jul 18 02:26:16.068860 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:16.068806    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp" podStartSLOduration=1.068788281 podStartE2EDuration="1.068788281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:26:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:26:16.068250196 +0000 UTC m=+2269.277964416" watchObservedRunningTime="2026-07-18 02:26:16.068788281 +0000 UTC m=+2269.278502503"
Jul 18 02:26:17.059219 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:17.059182    2580 generic.go:356] "Generic (PLEG): container finished" podID="8e388945-71ba-4ed3-a7e5-e4065c1b038f" containerID="2cfb7c8e1b9d4b4b2f34bdb25dbb3d4a87839bea3d42adc62665c437a3c17a39" exitCode=0
Jul 18 02:26:17.059598 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:17.059272    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp" event={"ID":"8e388945-71ba-4ed3-a7e5-e4065c1b038f","Type":"ContainerDied","Data":"2cfb7c8e1b9d4b4b2f34bdb25dbb3d4a87839bea3d42adc62665c437a3c17a39"}
Jul 18 02:26:18.189133 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:18.189111    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"
Jul 18 02:26:18.283264 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:18.283237    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e388945-71ba-4ed3-a7e5-e4065c1b038f-maas-api-service-ca\") pod \"8e388945-71ba-4ed3-a7e5-e4065c1b038f\" (UID: \"8e388945-71ba-4ed3-a7e5-e4065c1b038f\") "
Jul 18 02:26:18.283626 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:18.283606    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e388945-71ba-4ed3-a7e5-e4065c1b038f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8e388945-71ba-4ed3-a7e5-e4065c1b038f" (UID: "8e388945-71ba-4ed3-a7e5-e4065c1b038f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:26:18.384414 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:18.384346    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e388945-71ba-4ed3-a7e5-e4065c1b038f-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:26:19.070375 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:19.070343    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"
Jul 18 02:26:19.070544 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:19.070377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp" event={"ID":"8e388945-71ba-4ed3-a7e5-e4065c1b038f","Type":"ContainerDied","Data":"cecdadad1f0fb4e1953840b4046ffd0df5cbbdc1aba7f1cae8eb2003e660ec69"}
Jul 18 02:26:19.070544 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:19.070423    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cecdadad1f0fb4e1953840b4046ffd0df5cbbdc1aba7f1cae8eb2003e660ec69"
Jul 18 02:26:27.543325 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:27.543295    2580 scope.go:117] "RemoveContainer" containerID="9b7dd646c48fb3cb5963038935075bf02827e95523ebbc2bed29dcf867b9e8ff"
Jul 18 02:26:30.697966 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:30.697932    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"]
Jul 18 02:26:30.698463 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:30.698215    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8" podUID="c9be942f-b493-4895-bc17-0e4b925d5570" containerName="maas-api" containerID="cri-o://94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2" gracePeriod=30
Jul 18 02:26:30.732768 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:30.732741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:30.937548 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:30.937524    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:26:31.093861 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.093784    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9be942f-b493-4895-bc17-0e4b925d5570-maas-api-tls\") pod \"c9be942f-b493-4895-bc17-0e4b925d5570\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") "
Jul 18 02:26:31.093861 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.093843    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9be942f-b493-4895-bc17-0e4b925d5570-openshift-service-ca\") pod \"c9be942f-b493-4895-bc17-0e4b925d5570\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") "
Jul 18 02:26:31.094031 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.093916    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hdwn\" (UniqueName: \"kubernetes.io/projected/c9be942f-b493-4895-bc17-0e4b925d5570-kube-api-access-7hdwn\") pod \"c9be942f-b493-4895-bc17-0e4b925d5570\" (UID: \"c9be942f-b493-4895-bc17-0e4b925d5570\") "
Jul 18 02:26:31.094251 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.094226    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9be942f-b493-4895-bc17-0e4b925d5570-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9be942f-b493-4895-bc17-0e4b925d5570" (UID: "c9be942f-b493-4895-bc17-0e4b925d5570"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:26:31.096136 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.096110    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9be942f-b493-4895-bc17-0e4b925d5570-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9be942f-b493-4895-bc17-0e4b925d5570" (UID: "c9be942f-b493-4895-bc17-0e4b925d5570"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:26:31.096234 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.096190    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9be942f-b493-4895-bc17-0e4b925d5570-kube-api-access-7hdwn" (OuterVolumeSpecName: "kube-api-access-7hdwn") pod "c9be942f-b493-4895-bc17-0e4b925d5570" (UID: "c9be942f-b493-4895-bc17-0e4b925d5570"). InnerVolumeSpecName "kube-api-access-7hdwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:26:31.129940 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.129911    2580 generic.go:356] "Generic (PLEG): container finished" podID="c9be942f-b493-4895-bc17-0e4b925d5570" containerID="94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2" exitCode=0
Jul 18 02:26:31.130029 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.129959    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"
Jul 18 02:26:31.130029 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.129971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8" event={"ID":"c9be942f-b493-4895-bc17-0e4b925d5570","Type":"ContainerDied","Data":"94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2"}
Jul 18 02:26:31.130029 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.130014    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8" event={"ID":"c9be942f-b493-4895-bc17-0e4b925d5570","Type":"ContainerDied","Data":"e0db8f2b78626584246e26c07da0cff756ff4929da76e35d619b28035e1237e7"}
Jul 18 02:26:31.130155 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.130066    2580 scope.go:117] "RemoveContainer" containerID="94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2"
Jul 18 02:26:31.138586 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.138571    2580 scope.go:117] "RemoveContainer" containerID="94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2"
Jul 18 02:26:31.138844 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:26:31.138825    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2\": container with ID starting with 94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2 not found: ID does not exist" containerID="94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2"
Jul 18 02:26:31.138890 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.138860    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2"} err="failed to get container status \"94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2\": rpc error: code = NotFound desc = could not find container \"94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2\": container with ID starting with 94c37ac65180491f56e17b1c546b13962f690bd1daa965abe7b6988eaf9a9fb2 not found: ID does not exist"
Jul 18 02:26:31.152691 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.152669    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"]
Jul 18 02:26:31.154778 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.154762    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0e9c76-69dc67b5cb-84fc8"]
Jul 18 02:26:31.195271 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.195247    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hdwn\" (UniqueName: \"kubernetes.io/projected/c9be942f-b493-4895-bc17-0e4b925d5570-kube-api-access-7hdwn\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:26:31.195271 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.195269    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9be942f-b493-4895-bc17-0e4b925d5570-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:26:31.195383 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.195280    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9be942f-b493-4895-bc17-0e4b925d5570-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:26:31.392017 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:31.391934    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9be942f-b493-4895-bc17-0e4b925d5570" path="/var/lib/kubelet/pods/c9be942f-b493-4895-bc17-0e4b925d5570/volumes"
Jul 18 02:26:35.246034 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:35.245980    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:35.248067 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:35.248027    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"]
Jul 18 02:26:35.249121 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:35.249100    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9c76-g4jwp"]
Jul 18 02:26:35.390722 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:35.390690    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e388945-71ba-4ed3-a7e5-e4065c1b038f" path="/var/lib/kubelet/pods/8e388945-71ba-4ed3-a7e5-e4065c1b038f/volumes"
Jul 18 02:26:35.626963 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:35.626833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:41.365260 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:41.365220    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:41.427321 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:41.427283    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:41.623158 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:41.623058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:41.841351 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:41.841313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:42.175473 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:42.175442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:42.481784 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:42.481690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:42.580511 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:42.580480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:45.349734 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.349690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:45.431438 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.431395    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"]
Jul 18 02:26:45.431758 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.431743    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e388945-71ba-4ed3-a7e5-e4065c1b038f" containerName="revoke-keys"
Jul 18 02:26:45.431758 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.431757    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e388945-71ba-4ed3-a7e5-e4065c1b038f" containerName="revoke-keys"
Jul 18 02:26:45.431929 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.431769    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9be942f-b493-4895-bc17-0e4b925d5570" containerName="maas-api"
Jul 18 02:26:45.431929 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.431775    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9be942f-b493-4895-bc17-0e4b925d5570" containerName="maas-api"
Jul 18 02:26:45.431929 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.431827    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e388945-71ba-4ed3-a7e5-e4065c1b038f" containerName="revoke-keys"
Jul 18 02:26:45.431929 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.431834    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9be942f-b493-4895-bc17-0e4b925d5570" containerName="maas-api"
Jul 18 02:26:45.435213 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.435182    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:45.436584 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.436565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:45.440337 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.440320    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d05e31-serving-cert\""
Jul 18 02:26:45.440435 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.440347    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:26:45.440879 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.440861    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:26:45.440965 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.440898    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\""
Jul 18 02:26:45.447705 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.447661    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"]
Jul 18 02:26:45.519189 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.519151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:45.522235 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.522217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7vp2\" (UniqueName: \"kubernetes.io/projected/75b815e4-aef6-416a-a7ce-e787c9de5b25-kube-api-access-s7vp2\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:45.522309 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.522249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b815e4-aef6-416a-a7ce-e787c9de5b25-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:45.522374 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.522352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b815e4-aef6-416a-a7ce-e787c9de5b25-maas-api-tls\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:45.623116 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.623026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b815e4-aef6-416a-a7ce-e787c9de5b25-maas-api-tls\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:45.623116 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.623100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7vp2\" (UniqueName: \"kubernetes.io/projected/75b815e4-aef6-416a-a7ce-e787c9de5b25-kube-api-access-s7vp2\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:45.623314 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.623126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b815e4-aef6-416a-a7ce-e787c9de5b25-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:45.623314 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:26:45.623156    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-serving-cert: secret "maas-api-e2e-shared-a-d05e31-serving-cert" not found
Jul 18 02:26:45.623314 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:26:45.623240    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75b815e4-aef6-416a-a7ce-e787c9de5b25-maas-api-tls podName:75b815e4-aef6-416a-a7ce-e787c9de5b25 nodeName:}" failed. No retries permitted until 2026-07-18 02:26:46.123218821 +0000 UTC m=+2299.332933018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/75b815e4-aef6-416a-a7ce-e787c9de5b25-maas-api-tls") pod "maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4" (UID: "75b815e4-aef6-416a-a7ce-e787c9de5b25") : secret "maas-api-e2e-shared-a-d05e31-serving-cert" not found
Jul 18 02:26:45.623648 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.623632    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b815e4-aef6-416a-a7ce-e787c9de5b25-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:45.634126 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.634108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7vp2\" (UniqueName: \"kubernetes.io/projected/75b815e4-aef6-416a-a7ce-e787c9de5b25-kube-api-access-s7vp2\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:45.958037 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:45.958008    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:46.127429 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:46.127399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b815e4-aef6-416a-a7ce-e787c9de5b25-maas-api-tls\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:46.129917 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:46.129888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b815e4-aef6-416a-a7ce-e787c9de5b25-maas-api-tls\") pod \"maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:46.152360 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:46.152331    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:46.251165 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:46.251080    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:46.347405 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:46.347381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:46.480717 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:46.480694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"]
Jul 18 02:26:46.482604 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:26:46.482578    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b815e4_aef6_416a_a7ce_e787c9de5b25.slice/crio-b45bbaa12628d19f67892964c2b485c284e2715bfc28eb1195d4fc05e3835155 WatchSource:0}: Error finding container b45bbaa12628d19f67892964c2b485c284e2715bfc28eb1195d4fc05e3835155: Status 404 returned error can't find the container with id b45bbaa12628d19f67892964c2b485c284e2715bfc28eb1195d4fc05e3835155
Jul 18 02:26:47.212509 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:47.212472    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4" event={"ID":"75b815e4-aef6-416a-a7ce-e787c9de5b25","Type":"ContainerStarted","Data":"87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68"}
Jul 18 02:26:47.212672 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:47.212540    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:47.212672 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:47.212570    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4" event={"ID":"75b815e4-aef6-416a-a7ce-e787c9de5b25","Type":"ContainerStarted","Data":"b45bbaa12628d19f67892964c2b485c284e2715bfc28eb1195d4fc05e3835155"}
Jul 18 02:26:47.231008 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:47.230958    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4" podStartSLOduration=1.825617352 podStartE2EDuration="2.230941958s" totalImagesPullingTime="405.324606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:26:47.229870274 +0000 UTC m=+2300.439584496" watchObservedRunningTime="2026-07-18 02:26:47.230941958 +0000 UTC m=+2300.440656209"
Jul 18 02:26:49.836465 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:49.836436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:50.156119 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:50.156015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:50.450526 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:50.450487    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:50.737720 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:50.737636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:53.052906 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.052873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:53.151755 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.151719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:53.221642 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.221621    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:26:53.447197 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.447164    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"]
Jul 18 02:26:53.450772 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.450753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:53.453729 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.453712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-f0c43e-serving-cert\""
Jul 18 02:26:53.470496 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.470469    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"]
Jul 18 02:26:53.494139 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.494114    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:53.494423 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.494400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d1c081-a7d8-4b0d-ac31-2039edccfb34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:53.494496 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.494471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj9qr\" (UniqueName: \"kubernetes.io/projected/49d1c081-a7d8-4b0d-ac31-2039edccfb34-kube-api-access-mj9qr\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:53.494660 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.494630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d1c081-a7d8-4b0d-ac31-2039edccfb34-maas-api-tls\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:53.595262 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.595227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d1c081-a7d8-4b0d-ac31-2039edccfb34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:53.595418 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.595274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj9qr\" (UniqueName: \"kubernetes.io/projected/49d1c081-a7d8-4b0d-ac31-2039edccfb34-kube-api-access-mj9qr\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:53.595418 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.595306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d1c081-a7d8-4b0d-ac31-2039edccfb34-maas-api-tls\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:53.595418 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:26:53.595394    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-serving-cert: secret "maas-api-e2e-shared-b-f0c43e-serving-cert" not found
Jul 18 02:26:53.595523 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:26:53.595450    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49d1c081-a7d8-4b0d-ac31-2039edccfb34-maas-api-tls podName:49d1c081-a7d8-4b0d-ac31-2039edccfb34 nodeName:}" failed. No retries permitted until 2026-07-18 02:26:54.095435506 +0000 UTC m=+2307.305149703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49d1c081-a7d8-4b0d-ac31-2039edccfb34-maas-api-tls") pod "maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7" (UID: "49d1c081-a7d8-4b0d-ac31-2039edccfb34") : secret "maas-api-e2e-shared-b-f0c43e-serving-cert" not found
Jul 18 02:26:53.595822 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.595803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d1c081-a7d8-4b0d-ac31-2039edccfb34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:53.604317 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:53.604295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj9qr\" (UniqueName: \"kubernetes.io/projected/49d1c081-a7d8-4b0d-ac31-2039edccfb34-kube-api-access-mj9qr\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:54.047453 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:54.047418    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:54.099122 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:54.099092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d1c081-a7d8-4b0d-ac31-2039edccfb34-maas-api-tls\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:54.101744 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:54.101720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d1c081-a7d8-4b0d-ac31-2039edccfb34-maas-api-tls\") pod \"maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:54.148921 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:54.148892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:54.361222 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:54.361131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:54.494868 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:54.494833    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"]
Jul 18 02:26:54.497079 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:26:54.497033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d1c081_a7d8_4b0d_ac31_2039edccfb34.slice/crio-6e08472d6fe17447365769d4fe61bdf0864505f5ecb16d3596070af5bbb12f0b WatchSource:0}: Error finding container 6e08472d6fe17447365769d4fe61bdf0864505f5ecb16d3596070af5bbb12f0b: Status 404 returned error can't find the container with id 6e08472d6fe17447365769d4fe61bdf0864505f5ecb16d3596070af5bbb12f0b
Jul 18 02:26:55.254249 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:55.254214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7" event={"ID":"49d1c081-a7d8-4b0d-ac31-2039edccfb34","Type":"ContainerStarted","Data":"b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e"}
Jul 18 02:26:55.254620 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:55.254264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7" event={"ID":"49d1c081-a7d8-4b0d-ac31-2039edccfb34","Type":"ContainerStarted","Data":"6e08472d6fe17447365769d4fe61bdf0864505f5ecb16d3596070af5bbb12f0b"}
Jul 18 02:26:55.254620 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:55.254291    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:26:55.273156 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:55.273112    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7" podStartSLOduration=1.98288595 podStartE2EDuration="2.27309498s" totalImagesPullingTime="290.20903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:26:55.2713915 +0000 UTC m=+2308.481105720" watchObservedRunningTime="2026-07-18 02:26:55.27309498 +0000 UTC m=+2308.482809200"
Jul 18 02:26:57.516398 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:26:57.516368    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:01.264124 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:01.264095    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:27:07.848515 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:07.848479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:07.947984 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:07.947922    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:15.849198 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:15.849142    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:17.219152 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:17.219072    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:19.905883 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:19.905851    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"]
Jul 18 02:27:19.909352 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:19.909336    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:19.912298 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:19.912277    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"openshift-service-ca.crt\""
Jul 18 02:27:19.913577 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:19.913552    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"auth-test-model-d05e31-kserve-self-signed-certs\""
Jul 18 02:27:19.913687 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:19.913636    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"kube-root-ca.crt\""
Jul 18 02:27:19.913687 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:19.913655    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"default-dockercfg-vh9tm\""
Jul 18 02:27:19.919163 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:19.919142    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"]
Jul 18 02:27:19.930110 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:19.930085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:20.036511 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.036479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-model-cache\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.036511 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.036513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjqcj\" (UniqueName: \"kubernetes.io/projected/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-kube-api-access-hjqcj\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.036712 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.036547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tls-certs\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.036712 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.036601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-home\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.036712 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.036649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-dshm\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.036712 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.036666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tmp-dir\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.137397 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.137365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-model-cache\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.137516 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.137403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjqcj\" (UniqueName: \"kubernetes.io/projected/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-kube-api-access-hjqcj\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.137516 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.137447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tls-certs\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.137516 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.137488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-home\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.137628 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.137521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-dshm\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.137628 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.137541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tmp-dir\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.137869 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.137850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-model-cache\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.137907 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.137872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-home\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.137907 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.137897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tmp-dir\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.139954 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.139929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-dshm\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.140187 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.140166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tls-certs\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.145497 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.145474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjqcj\" (UniqueName: \"kubernetes.io/projected/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-kube-api-access-hjqcj\") pod \"auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") " pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.218689 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.218649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:20.348798 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.348773    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"]
Jul 18 02:27:20.351312 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:27:20.351286    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfad056ef_1b4e_47cc_9e2f_9362c255b2f0.slice/crio-a342ba28078ed2c9e44af766e3eef2d2d06a6a0de29fd4fad45891b5e50e8cc3 WatchSource:0}: Error finding container a342ba28078ed2c9e44af766e3eef2d2d06a6a0de29fd4fad45891b5e50e8cc3: Status 404 returned error can't find the container with id a342ba28078ed2c9e44af766e3eef2d2d06a6a0de29fd4fad45891b5e50e8cc3
Jul 18 02:27:20.377337 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:20.377313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb" event={"ID":"fad056ef-1b4e-47cc-9e2f-9362c255b2f0","Type":"ContainerStarted","Data":"a342ba28078ed2c9e44af766e3eef2d2d06a6a0de29fd4fad45891b5e50e8cc3"}
Jul 18 02:27:21.383826 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:21.383787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb" event={"ID":"fad056ef-1b4e-47cc-9e2f-9362c255b2f0","Type":"ContainerStarted","Data":"59bae5496b1895cd998a3c22eccb78d7cb4a197506ded10af31781fc08b0b0d0"}
Jul 18 02:27:21.401005 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:21.400953    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb" podStartSLOduration=2.400938028 podStartE2EDuration="2.400938028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:27:21.39930681 +0000 UTC m=+2334.609021030" watchObservedRunningTime="2026-07-18 02:27:21.400938028 +0000 UTC m=+2334.610652247"
Jul 18 02:27:27.572007 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:27.571976    2580 scope.go:117] "RemoveContainer" containerID="510d26cf283a4d12d617b9bb9fab13786e733b132ea9b231329686c91fb1ea07"
Jul 18 02:27:30.218893 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:30.218849    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:30.219285 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:30.218909    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:30.231133 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:30.231108    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:30.438084 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:30.438056    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:27:48.299227 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.299197    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"]
Jul 18 02:27:48.302764 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.302744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.305516 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.305492    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"kube-root-ca.crt\""
Jul 18 02:27:48.305661 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.305644    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"openshift-service-ca.crt\""
Jul 18 02:27:48.306542 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.306524    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"default-dockercfg-6fxsl\""
Jul 18 02:27:48.306667 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.306560    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"auth-test-model-f0c43e-kserve-self-signed-certs\""
Jul 18 02:27:48.314291 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.314271    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"]
Jul 18 02:27:48.321835 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.321810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:48.386728 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.386692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-model-cache\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.386928 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.386737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxsbm\" (UniqueName: \"kubernetes.io/projected/f3dc2474-06df-4bbe-acd4-1bd5da985a30-kube-api-access-bxsbm\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.386928 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.386769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tls-certs\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.386928 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.386823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-dshm\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.386928 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.386843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tmp-dir\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.387091 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.386943    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-home\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.487384 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.487357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-home\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.487503 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.487405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-model-cache\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.487503 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.487432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxsbm\" (UniqueName: \"kubernetes.io/projected/f3dc2474-06df-4bbe-acd4-1bd5da985a30-kube-api-access-bxsbm\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.487503 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.487483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tls-certs\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.487680 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.487540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-dshm\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.487680 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.487565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tmp-dir\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.487787 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.487727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-home\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.487844 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.487813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-model-cache\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.488007 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.487982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tmp-dir\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.489878 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.489851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-dshm\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.490156 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.490137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tls-certs\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.495449 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.495431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxsbm\" (UniqueName: \"kubernetes.io/projected/f3dc2474-06df-4bbe-acd4-1bd5da985a30-kube-api-access-bxsbm\") pod \"auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") " pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.612360 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.612277    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:48.747739 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:48.747717    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"]
Jul 18 02:27:48.749692 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:27:48.749660    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3dc2474_06df_4bbe_acd4_1bd5da985a30.slice/crio-b92792f7e4d80dd1df4de0402dd145b6dd93e99989a3864c4812e734e11c18c3 WatchSource:0}: Error finding container b92792f7e4d80dd1df4de0402dd145b6dd93e99989a3864c4812e734e11c18c3: Status 404 returned error can't find the container with id b92792f7e4d80dd1df4de0402dd145b6dd93e99989a3864c4812e734e11c18c3
Jul 18 02:27:49.517259 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:49.517222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq" event={"ID":"f3dc2474-06df-4bbe-acd4-1bd5da985a30","Type":"ContainerStarted","Data":"93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc"}
Jul 18 02:27:49.517664 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:49.517274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq" event={"ID":"f3dc2474-06df-4bbe-acd4-1bd5da985a30","Type":"ContainerStarted","Data":"b92792f7e4d80dd1df4de0402dd145b6dd93e99989a3864c4812e734e11c18c3"}
Jul 18 02:27:49.535687 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:49.535643    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq" podStartSLOduration=1.53562865 podStartE2EDuration="1.53562865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:27:49.534273338 +0000 UTC m=+2362.743987558" watchObservedRunningTime="2026-07-18 02:27:49.53562865 +0000 UTC m=+2362.745342867"
Jul 18 02:27:55.473176 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:55.473138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:55.574005 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:55.573943    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:58.613298 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:58.613251    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:58.613694 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:58.613324    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:58.625972 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:58.625948    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:27:59.584064 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:27:59.584012    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:28:20.990202 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:28:20.990170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:28:21.049484 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:28:21.049448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:29:01.012381 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:29:01.012294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:29:37.977315 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:29:37.977274    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:29:55.978844 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:29:55.978810    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"]
Jul 18 02:29:55.997363 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:29:55.997329    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:29:56.782239 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:29:56.782205    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"]
Jul 18 02:29:56.798520 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:29:56.798486    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:30:00.139262 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.139226    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl"]
Jul 18 02:30:00.142951 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.142928    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl"
Jul 18 02:30:00.145508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.145486    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:30:00.149384 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.149357    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm"]
Jul 18 02:30:00.152902 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.152884    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm"
Jul 18 02:30:00.153688 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.153668    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl"
Jul 18 02:30:00.154096 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.154074    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl"]
Jul 18 02:30:00.166785 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.166469    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm"]
Jul 18 02:30:00.167246 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.167227    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm"
Jul 18 02:30:00.315475 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.315447    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl"]
Jul 18 02:30:00.318564 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:30:00.318538    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30eab2ad_2643_487b_80ed_d6e868803ad7.slice/crio-ac4864f5939006ae7a408d0d1556e7d027438c15c95fe9f90628f0dda1a80207 WatchSource:0}: Error finding container ac4864f5939006ae7a408d0d1556e7d027438c15c95fe9f90628f0dda1a80207: Status 404 returned error can't find the container with id ac4864f5939006ae7a408d0d1556e7d027438c15c95fe9f90628f0dda1a80207
Jul 18 02:30:00.338511 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:00.338493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm"]
Jul 18 02:30:00.340552 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:30:00.340525    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b1c46c_a0e7_4d73_b544_f1051409df12.slice/crio-8be95521c5e1a1fb78ad10446241cadc2a6191686e9464b596f9d12b865536dd WatchSource:0}: Error finding container 8be95521c5e1a1fb78ad10446241cadc2a6191686e9464b596f9d12b865536dd: Status 404 returned error can't find the container with id 8be95521c5e1a1fb78ad10446241cadc2a6191686e9464b596f9d12b865536dd
Jul 18 02:30:01.173876 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:01.173841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl" event={"ID":"30eab2ad-2643-487b-80ed-d6e868803ad7","Type":"ContainerStarted","Data":"bb5b9e6b5e0dc7e351a609073ba0fb68b2b7b8a9307fa8afd03fda415da4a6f8"}
Jul 18 02:30:01.174327 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:01.173895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl" event={"ID":"30eab2ad-2643-487b-80ed-d6e868803ad7","Type":"ContainerStarted","Data":"ac4864f5939006ae7a408d0d1556e7d027438c15c95fe9f90628f0dda1a80207"}
Jul 18 02:30:01.175679 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:01.175650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm" event={"ID":"e0b1c46c-a0e7-4d73-b544-f1051409df12","Type":"ContainerStarted","Data":"8a1402cd8df8091cae7f4141b15acb36c587aebc10bc0562637777ebc2ac645d"}
Jul 18 02:30:01.175831 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:01.175695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm" event={"ID":"e0b1c46c-a0e7-4d73-b544-f1051409df12","Type":"ContainerStarted","Data":"8be95521c5e1a1fb78ad10446241cadc2a6191686e9464b596f9d12b865536dd"}
Jul 18 02:30:01.188985 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:01.188939    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl" podStartSLOduration=1.188925433 podStartE2EDuration="1.188925433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:30:01.187693168 +0000 UTC m=+2494.397407389" watchObservedRunningTime="2026-07-18 02:30:01.188925433 +0000 UTC m=+2494.398639690"
Jul 18 02:30:01.203184 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:01.203148    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm" podStartSLOduration=1.203131232 podStartE2EDuration="1.203131232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:30:01.20131388 +0000 UTC m=+2494.411028100" watchObservedRunningTime="2026-07-18 02:30:01.203131232 +0000 UTC m=+2494.412845453"
Jul 18 02:30:02.182031 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:02.181994    2580 generic.go:356] "Generic (PLEG): container finished" podID="e0b1c46c-a0e7-4d73-b544-f1051409df12" containerID="8a1402cd8df8091cae7f4141b15acb36c587aebc10bc0562637777ebc2ac645d" exitCode=0
Jul 18 02:30:02.182532 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:02.182097    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm" event={"ID":"e0b1c46c-a0e7-4d73-b544-f1051409df12","Type":"ContainerDied","Data":"8a1402cd8df8091cae7f4141b15acb36c587aebc10bc0562637777ebc2ac645d"}
Jul 18 02:30:02.183607 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:02.183585    2580 generic.go:356] "Generic (PLEG): container finished" podID="30eab2ad-2643-487b-80ed-d6e868803ad7" containerID="bb5b9e6b5e0dc7e351a609073ba0fb68b2b7b8a9307fa8afd03fda415da4a6f8" exitCode=0
Jul 18 02:30:02.183727 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:02.183624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl" event={"ID":"30eab2ad-2643-487b-80ed-d6e868803ad7","Type":"ContainerDied","Data":"bb5b9e6b5e0dc7e351a609073ba0fb68b2b7b8a9307fa8afd03fda415da4a6f8"}
Jul 18 02:30:03.331621 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:03.331594    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl"
Jul 18 02:30:03.334872 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:03.334841    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm"
Jul 18 02:30:04.194306 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:04.194275    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm"
Jul 18 02:30:04.194306 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:04.194293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm" event={"ID":"e0b1c46c-a0e7-4d73-b544-f1051409df12","Type":"ContainerDied","Data":"8be95521c5e1a1fb78ad10446241cadc2a6191686e9464b596f9d12b865536dd"}
Jul 18 02:30:04.194532 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:04.194369    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8be95521c5e1a1fb78ad10446241cadc2a6191686e9464b596f9d12b865536dd"
Jul 18 02:30:04.195626 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:04.195604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl" event={"ID":"30eab2ad-2643-487b-80ed-d6e868803ad7","Type":"ContainerDied","Data":"ac4864f5939006ae7a408d0d1556e7d027438c15c95fe9f90628f0dda1a80207"}
Jul 18 02:30:04.195626 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:04.195618    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl"
Jul 18 02:30:04.195835 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:04.195636    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac4864f5939006ae7a408d0d1556e7d027438c15c95fe9f90628f0dda1a80207"
Jul 18 02:30:05.229799 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:05.229742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:30:11.000010 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:10.999958    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb" podUID="fad056ef-1b4e-47cc-9e2f-9362c255b2f0" containerName="main" containerID="cri-o://59bae5496b1895cd998a3c22eccb78d7cb4a197506ded10af31781fc08b0b0d0" gracePeriod=45
Jul 18 02:30:11.233568 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.233538    2580 generic.go:356] "Generic (PLEG): container finished" podID="fad056ef-1b4e-47cc-9e2f-9362c255b2f0" containerID="59bae5496b1895cd998a3c22eccb78d7cb4a197506ded10af31781fc08b0b0d0" exitCode=0
Jul 18 02:30:11.233719 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.233615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb" event={"ID":"fad056ef-1b4e-47cc-9e2f-9362c255b2f0","Type":"ContainerDied","Data":"59bae5496b1895cd998a3c22eccb78d7cb4a197506ded10af31781fc08b0b0d0"}
Jul 18 02:30:11.256765 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.256708    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:30:11.332935 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.332900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-home\") pod \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") "
Jul 18 02:30:11.333124 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.333012    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tls-certs\") pod \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") "
Jul 18 02:30:11.333124 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.333065    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-dshm\") pod \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") "
Jul 18 02:30:11.333124 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.333108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjqcj\" (UniqueName: \"kubernetes.io/projected/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-kube-api-access-hjqcj\") pod \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") "
Jul 18 02:30:11.333266 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.333143    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tmp-dir\") pod \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") "
Jul 18 02:30:11.333266 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.333175    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-model-cache\") pod \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\" (UID: \"fad056ef-1b4e-47cc-9e2f-9362c255b2f0\") "
Jul 18 02:30:11.333266 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.333244    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-home" (OuterVolumeSpecName: "home") pod "fad056ef-1b4e-47cc-9e2f-9362c255b2f0" (UID: "fad056ef-1b4e-47cc-9e2f-9362c255b2f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:11.333473 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.333452    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-home\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:11.333543 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.333455    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fad056ef-1b4e-47cc-9e2f-9362c255b2f0" (UID: "fad056ef-1b4e-47cc-9e2f-9362c255b2f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:11.333543 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.333480    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-model-cache" (OuterVolumeSpecName: "model-cache") pod "fad056ef-1b4e-47cc-9e2f-9362c255b2f0" (UID: "fad056ef-1b4e-47cc-9e2f-9362c255b2f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:11.335497 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.335470    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-kube-api-access-hjqcj" (OuterVolumeSpecName: "kube-api-access-hjqcj") pod "fad056ef-1b4e-47cc-9e2f-9362c255b2f0" (UID: "fad056ef-1b4e-47cc-9e2f-9362c255b2f0"). InnerVolumeSpecName "kube-api-access-hjqcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:30:11.335497 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.335480    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-dshm" (OuterVolumeSpecName: "dshm") pod "fad056ef-1b4e-47cc-9e2f-9362c255b2f0" (UID: "fad056ef-1b4e-47cc-9e2f-9362c255b2f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:11.335659 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.335509    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fad056ef-1b4e-47cc-9e2f-9362c255b2f0" (UID: "fad056ef-1b4e-47cc-9e2f-9362c255b2f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:30:11.434393 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.434361    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tls-certs\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:11.434393 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.434388    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-dshm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:11.434393 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.434398    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjqcj\" (UniqueName: \"kubernetes.io/projected/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-kube-api-access-hjqcj\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:11.434597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.434407    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-tmp-dir\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:11.434597 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.434415    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad056ef-1b4e-47cc-9e2f-9362c255b2f0-model-cache\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:11.803739 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:11.803693    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq" podUID="f3dc2474-06df-4bbe-acd4-1bd5da985a30" containerName="main" containerID="cri-o://93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc" gracePeriod=45
Jul 18 02:30:12.068368 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.068301    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:30:12.140403 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140368    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-dshm\") pod \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") "
Jul 18 02:30:12.140591 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140421    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-model-cache\") pod \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") "
Jul 18 02:30:12.140591 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140441    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxsbm\" (UniqueName: \"kubernetes.io/projected/f3dc2474-06df-4bbe-acd4-1bd5da985a30-kube-api-access-bxsbm\") pod \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") "
Jul 18 02:30:12.140591 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140499    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-home\") pod \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") "
Jul 18 02:30:12.140591 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tls-certs\") pod \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") "
Jul 18 02:30:12.140591 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tmp-dir\") pod \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\" (UID: \"f3dc2474-06df-4bbe-acd4-1bd5da985a30\") "
Jul 18 02:30:12.140860 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140710    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3dc2474-06df-4bbe-acd4-1bd5da985a30" (UID: "f3dc2474-06df-4bbe-acd4-1bd5da985a30"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.140860 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140835    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-home" (OuterVolumeSpecName: "home") pod "f3dc2474-06df-4bbe-acd4-1bd5da985a30" (UID: "f3dc2474-06df-4bbe-acd4-1bd5da985a30"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.140948 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3dc2474-06df-4bbe-acd4-1bd5da985a30" (UID: "f3dc2474-06df-4bbe-acd4-1bd5da985a30"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.140948 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.140872    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-model-cache\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.142889 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.142862    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-dshm" (OuterVolumeSpecName: "dshm") pod "f3dc2474-06df-4bbe-acd4-1bd5da985a30" (UID: "f3dc2474-06df-4bbe-acd4-1bd5da985a30"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.142889 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.142862    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3dc2474-06df-4bbe-acd4-1bd5da985a30" (UID: "f3dc2474-06df-4bbe-acd4-1bd5da985a30"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:30:12.143062 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.142912    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3dc2474-06df-4bbe-acd4-1bd5da985a30-kube-api-access-bxsbm" (OuterVolumeSpecName: "kube-api-access-bxsbm") pod "f3dc2474-06df-4bbe-acd4-1bd5da985a30" (UID: "f3dc2474-06df-4bbe-acd4-1bd5da985a30"). InnerVolumeSpecName "kube-api-access-bxsbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:30:12.239916 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.239887    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"
Jul 18 02:30:12.240156 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.239922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb" event={"ID":"fad056ef-1b4e-47cc-9e2f-9362c255b2f0","Type":"ContainerDied","Data":"a342ba28078ed2c9e44af766e3eef2d2d06a6a0de29fd4fad45891b5e50e8cc3"}
Jul 18 02:30:12.240156 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.239974    2580 scope.go:117] "RemoveContainer" containerID="59bae5496b1895cd998a3c22eccb78d7cb4a197506ded10af31781fc08b0b0d0"
Jul 18 02:30:12.241359 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.241336    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tls-certs\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.241359 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.241357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq" event={"ID":"f3dc2474-06df-4bbe-acd4-1bd5da985a30","Type":"ContainerDied","Data":"93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc"}
Jul 18 02:30:12.241508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.241371    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"
Jul 18 02:30:12.241508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.241363    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-tmp-dir\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.241508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.241402    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-dshm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.241508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.241413    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxsbm\" (UniqueName: \"kubernetes.io/projected/f3dc2474-06df-4bbe-acd4-1bd5da985a30-kube-api-access-bxsbm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.241508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.241422    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dc2474-06df-4bbe-acd4-1bd5da985a30-home\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.241508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.241331    2580 generic.go:356] "Generic (PLEG): container finished" podID="f3dc2474-06df-4bbe-acd4-1bd5da985a30" containerID="93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc" exitCode=0
Jul 18 02:30:12.241718 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.241542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq" event={"ID":"f3dc2474-06df-4bbe-acd4-1bd5da985a30","Type":"ContainerDied","Data":"b92792f7e4d80dd1df4de0402dd145b6dd93e99989a3864c4812e734e11c18c3"}
Jul 18 02:30:12.249906 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.249834    2580 scope.go:117] "RemoveContainer" containerID="93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc"
Jul 18 02:30:12.260497 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.260470    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"]
Jul 18 02:30:12.260563 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.260515    2580 scope.go:117] "RemoveContainer" containerID="93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc"
Jul 18 02:30:12.260796 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:30:12.260777    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc\": container with ID starting with 93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc not found: ID does not exist" containerID="93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc"
Jul 18 02:30:12.260859 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.260805    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc"} err="failed to get container status \"93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc\": rpc error: code = NotFound desc = could not find container \"93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc\": container with ID starting with 93dbed485bb440190fc9bace3afb895dc224c73b271d488076c6c9f34e3540fc not found: ID does not exist"
Jul 18 02:30:12.264234 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.264211    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/auth-test-model-d05e31-kserve-7974c9f9ff-bxhkb"]
Jul 18 02:30:12.277616 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.277593    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"]
Jul 18 02:30:12.281423 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:12.281403    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/auth-test-model-f0c43e-kserve-78b5b65d79-pxrbq"]
Jul 18 02:30:13.391408 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:13.391372    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3dc2474-06df-4bbe-acd4-1bd5da985a30" path="/var/lib/kubelet/pods/f3dc2474-06df-4bbe-acd4-1bd5da985a30/volumes"
Jul 18 02:30:13.391867 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:13.391808    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad056ef-1b4e-47cc-9e2f-9362c255b2f0" path="/var/lib/kubelet/pods/fad056ef-1b4e-47cc-9e2f-9362c255b2f0/volumes"
Jul 18 02:30:31.201678 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.201633    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"]
Jul 18 02:30:31.202327 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202306    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad056ef-1b4e-47cc-9e2f-9362c255b2f0" containerName="main"
Jul 18 02:30:31.202416 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202331    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad056ef-1b4e-47cc-9e2f-9362c255b2f0" containerName="main"
Jul 18 02:30:31.202416 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202346    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3dc2474-06df-4bbe-acd4-1bd5da985a30" containerName="main"
Jul 18 02:30:31.202416 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202353    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3dc2474-06df-4bbe-acd4-1bd5da985a30" containerName="main"
Jul 18 02:30:31.202416 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202385    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0b1c46c-a0e7-4d73-b544-f1051409df12" containerName="cleanup"
Jul 18 02:30:31.202416 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202393    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0b1c46c-a0e7-4d73-b544-f1051409df12" containerName="cleanup"
Jul 18 02:30:31.202416 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202410    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30eab2ad-2643-487b-80ed-d6e868803ad7" containerName="cleanup"
Jul 18 02:30:31.202710 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202418    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="30eab2ad-2643-487b-80ed-d6e868803ad7" containerName="cleanup"
Jul 18 02:30:31.202710 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202506    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3dc2474-06df-4bbe-acd4-1bd5da985a30" containerName="main"
Jul 18 02:30:31.202710 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202516    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0b1c46c-a0e7-4d73-b544-f1051409df12" containerName="cleanup"
Jul 18 02:30:31.202710 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202525    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="30eab2ad-2643-487b-80ed-d6e868803ad7" containerName="cleanup"
Jul 18 02:30:31.202710 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.202535    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad056ef-1b4e-47cc-9e2f-9362c255b2f0" containerName="main"
Jul 18 02:30:31.207379 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.207358    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.211820 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.211783    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"sub-test-model-f0c43e-kserve-self-signed-certs\""
Jul 18 02:30:31.211820 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.211801    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"default-dockercfg-6fxsl\""
Jul 18 02:30:31.211820 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.211789    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"openshift-service-ca.crt\""
Jul 18 02:30:31.212118 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.212067    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"kube-root-ca.crt\""
Jul 18 02:30:31.215127 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.214507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-dshm\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.215127 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.214625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-home\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.215127 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.214702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5nm5\" (UniqueName: \"kubernetes.io/projected/16849cd8-e418-4114-bcf1-d53b5e639562-kube-api-access-v5nm5\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.215127 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.214718    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"]
Jul 18 02:30:31.215127 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.214761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16849cd8-e418-4114-bcf1-d53b5e639562-tls-certs\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.215127 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.214801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-model-cache\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.215127 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.214834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-tmp-dir\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.293970 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.293941    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:30:31.315457 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.315426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-dshm\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.315575 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.315509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-home\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.315575 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.315544    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5nm5\" (UniqueName: \"kubernetes.io/projected/16849cd8-e418-4114-bcf1-d53b5e639562-kube-api-access-v5nm5\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.315657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.315586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16849cd8-e418-4114-bcf1-d53b5e639562-tls-certs\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.315657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.315619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-model-cache\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.315657 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.315650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-tmp-dir\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.315964 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.315943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-home\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.315964 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.315956    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-tmp-dir\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.316099 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.315990    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-model-cache\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.317946 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.317929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-dshm\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.318262 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.318241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16849cd8-e418-4114-bcf1-d53b5e639562-tls-certs\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.324751 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.324729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5nm5\" (UniqueName: \"kubernetes.io/projected/16849cd8-e418-4114-bcf1-d53b5e639562-kube-api-access-v5nm5\") pod \"sub-test-model-f0c43e-kserve-779d8756b5-rwzg2\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") " pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.517895 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.517807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:31.656666 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:31.656642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"]
Jul 18 02:30:31.658933 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:30:31.658905    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16849cd8_e418_4114_bcf1_d53b5e639562.slice/crio-2c3a5624d7dd6085ffd478cde6439bbdd94643c4b6df4ee54c8f7529524d8833 WatchSource:0}: Error finding container 2c3a5624d7dd6085ffd478cde6439bbdd94643c4b6df4ee54c8f7529524d8833: Status 404 returned error can't find the container with id 2c3a5624d7dd6085ffd478cde6439bbdd94643c4b6df4ee54c8f7529524d8833
Jul 18 02:30:32.349215 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:32.349179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2" event={"ID":"16849cd8-e418-4114-bcf1-d53b5e639562","Type":"ContainerStarted","Data":"aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3"}
Jul 18 02:30:32.349615 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:32.349221    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2" event={"ID":"16849cd8-e418-4114-bcf1-d53b5e639562","Type":"ContainerStarted","Data":"2c3a5624d7dd6085ffd478cde6439bbdd94643c4b6df4ee54c8f7529524d8833"}
Jul 18 02:30:32.368396 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:32.368336    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2" podStartSLOduration=1.36831821 podStartE2EDuration="1.36831821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:30:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:30:32.368031939 +0000 UTC m=+2525.577746160" watchObservedRunningTime="2026-07-18 02:30:32.36831821 +0000 UTC m=+2525.578032430"
Jul 18 02:30:41.518177 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:41.518139    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:41.518603 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:41.518218    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:41.531448 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:41.531422    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:30:42.411321 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:30:42.411294    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:31:13.528502 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:13.528466    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:31:14.278571 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:14.278537    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"]
Jul 18 02:31:14.298054 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:14.298016    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:31:27.624022 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:27.623993    2580 scope.go:117] "RemoveContainer" containerID="29cb326abf0dc682eff4833585ea456a50a6db6ec7c4b51030a89883834617b7"
Jul 18 02:31:27.632444 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:27.632424    2580 scope.go:117] "RemoveContainer" containerID="86723d954d9652d223b3d40bbda637f478ba5771e72dfff41f7303b7a81a5b93"
Jul 18 02:31:29.300851 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.300789    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2" podUID="16849cd8-e418-4114-bcf1-d53b5e639562" containerName="main" containerID="cri-o://aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3" gracePeriod=45
Jul 18 02:31:29.561761 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.561701    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:31:29.636475 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.636446    2580 generic.go:356] "Generic (PLEG): container finished" podID="16849cd8-e418-4114-bcf1-d53b5e639562" containerID="aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3" exitCode=0
Jul 18 02:31:29.636663 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.636496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2" event={"ID":"16849cd8-e418-4114-bcf1-d53b5e639562","Type":"ContainerDied","Data":"aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3"}
Jul 18 02:31:29.636663 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.636504    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"
Jul 18 02:31:29.636663 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.636523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2" event={"ID":"16849cd8-e418-4114-bcf1-d53b5e639562","Type":"ContainerDied","Data":"2c3a5624d7dd6085ffd478cde6439bbdd94643c4b6df4ee54c8f7529524d8833"}
Jul 18 02:31:29.636663 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.636540    2580 scope.go:117] "RemoveContainer" containerID="aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3"
Jul 18 02:31:29.638764 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.638748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5nm5\" (UniqueName: \"kubernetes.io/projected/16849cd8-e418-4114-bcf1-d53b5e639562-kube-api-access-v5nm5\") pod \"16849cd8-e418-4114-bcf1-d53b5e639562\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") "
Jul 18 02:31:29.638851 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.638785    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-tmp-dir\") pod \"16849cd8-e418-4114-bcf1-d53b5e639562\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") "
Jul 18 02:31:29.638922 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.638850    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16849cd8-e418-4114-bcf1-d53b5e639562-tls-certs\") pod \"16849cd8-e418-4114-bcf1-d53b5e639562\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") "
Jul 18 02:31:29.638922 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.638884    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-model-cache\") pod \"16849cd8-e418-4114-bcf1-d53b5e639562\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") "
Jul 18 02:31:29.639032 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.638938    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-home\") pod \"16849cd8-e418-4114-bcf1-d53b5e639562\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") "
Jul 18 02:31:29.639032 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.638961    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-dshm\") pod \"16849cd8-e418-4114-bcf1-d53b5e639562\" (UID: \"16849cd8-e418-4114-bcf1-d53b5e639562\") "
Jul 18 02:31:29.639166 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.639132    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "16849cd8-e418-4114-bcf1-d53b5e639562" (UID: "16849cd8-e418-4114-bcf1-d53b5e639562"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:31:29.639237 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.639206    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-home" (OuterVolumeSpecName: "home") pod "16849cd8-e418-4114-bcf1-d53b5e639562" (UID: "16849cd8-e418-4114-bcf1-d53b5e639562"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:31:29.639294 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.639270    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-tmp-dir\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:31:29.639349 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.639272    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-model-cache" (OuterVolumeSpecName: "model-cache") pod "16849cd8-e418-4114-bcf1-d53b5e639562" (UID: "16849cd8-e418-4114-bcf1-d53b5e639562"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:31:29.641213 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.641183    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16849cd8-e418-4114-bcf1-d53b5e639562-kube-api-access-v5nm5" (OuterVolumeSpecName: "kube-api-access-v5nm5") pod "16849cd8-e418-4114-bcf1-d53b5e639562" (UID: "16849cd8-e418-4114-bcf1-d53b5e639562"). InnerVolumeSpecName "kube-api-access-v5nm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:31:29.641309 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.641208    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16849cd8-e418-4114-bcf1-d53b5e639562-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "16849cd8-e418-4114-bcf1-d53b5e639562" (UID: "16849cd8-e418-4114-bcf1-d53b5e639562"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:31:29.641309 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.641239    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-dshm" (OuterVolumeSpecName: "dshm") pod "16849cd8-e418-4114-bcf1-d53b5e639562" (UID: "16849cd8-e418-4114-bcf1-d53b5e639562"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:31:29.658811 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.658790    2580 scope.go:117] "RemoveContainer" containerID="aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3"
Jul 18 02:31:29.659087 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:31:29.659069    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3\": container with ID starting with aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3 not found: ID does not exist" containerID="aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3"
Jul 18 02:31:29.659149 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.659094    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3"} err="failed to get container status \"aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3\": rpc error: code = NotFound desc = could not find container \"aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3\": container with ID starting with aea8137580b3caf37445ea774bcbfed27af2db91de7b0fa2514a399dc62ebff3 not found: ID does not exist"
Jul 18 02:31:29.739998 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.739970    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-home\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:31:29.739998 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.739993    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-dshm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:31:29.739998 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.740003    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5nm5\" (UniqueName: \"kubernetes.io/projected/16849cd8-e418-4114-bcf1-d53b5e639562-kube-api-access-v5nm5\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:31:29.740230 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.740012    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16849cd8-e418-4114-bcf1-d53b5e639562-tls-certs\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:31:29.740230 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.740021    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16849cd8-e418-4114-bcf1-d53b5e639562-model-cache\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:31:29.957967 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.957933    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"]
Jul 18 02:31:29.960320 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:29.960298    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/sub-test-model-f0c43e-kserve-779d8756b5-rwzg2"]
Jul 18 02:31:31.391452 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:31.391421    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16849cd8-e418-4114-bcf1-d53b5e639562" path="/var/lib/kubelet/pods/16849cd8-e418-4114-bcf1-d53b5e639562/volumes"
Jul 18 02:31:32.504249 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.504217    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"]
Jul 18 02:31:32.504675 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.504601    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16849cd8-e418-4114-bcf1-d53b5e639562" containerName="main"
Jul 18 02:31:32.504675 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.504615    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="16849cd8-e418-4114-bcf1-d53b5e639562" containerName="main"
Jul 18 02:31:32.504815 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.504726    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="16849cd8-e418-4114-bcf1-d53b5e639562" containerName="main"
Jul 18 02:31:32.509472 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.509449    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.513535 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.513483    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"default-dockercfg-vh9tm\""
Jul 18 02:31:32.513535 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.513524    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"rate-test-model-d05e31-kserve-self-signed-certs\""
Jul 18 02:31:32.513719 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.513562    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"kube-root-ca.crt\""
Jul 18 02:31:32.513719 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.513623    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"openshift-service-ca.crt\""
Jul 18 02:31:32.518229 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.518207    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"]
Jul 18 02:31:32.532528 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.532504    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:31:32.564484 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.564456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlms2\" (UniqueName: \"kubernetes.io/projected/83275195-3d7e-44ab-b4ef-54edac3cecfc-kube-api-access-wlms2\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.564595 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.564489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-tmp-dir\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.564644 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.564593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-model-cache\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.564644 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.564636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-dshm\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.564724 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.564667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83275195-3d7e-44ab-b4ef-54edac3cecfc-tls-certs\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.564724 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.564714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-home\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.665420 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.665392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlms2\" (UniqueName: \"kubernetes.io/projected/83275195-3d7e-44ab-b4ef-54edac3cecfc-kube-api-access-wlms2\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.665570 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.665425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-tmp-dir\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.665570 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.665479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-model-cache\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.665570 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.665500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-dshm\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.665570 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.665514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83275195-3d7e-44ab-b4ef-54edac3cecfc-tls-certs\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.665796 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.665616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-home\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.665944 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.665921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-tmp-dir\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.665944 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.665933    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-model-cache\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.666148 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.666024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-home\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.667974 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.667952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-dshm\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.668241 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.668225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83275195-3d7e-44ab-b4ef-54edac3cecfc-tls-certs\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.673479 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.673458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlms2\" (UniqueName: \"kubernetes.io/projected/83275195-3d7e-44ab-b4ef-54edac3cecfc-kube-api-access-wlms2\") pod \"rate-test-model-d05e31-kserve-7776659d99-ndrm4\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") " pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.819749 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.819664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:32.952834 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:32.952808    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"]
Jul 18 02:31:32.954591 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:31:32.954560    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83275195_3d7e_44ab_b4ef_54edac3cecfc.slice/crio-b1e7ead19d0d5a1202ea1cdb5a79f521a707432d869bc948418b3205b34748ae WatchSource:0}: Error finding container b1e7ead19d0d5a1202ea1cdb5a79f521a707432d869bc948418b3205b34748ae: Status 404 returned error can't find the container with id b1e7ead19d0d5a1202ea1cdb5a79f521a707432d869bc948418b3205b34748ae
Jul 18 02:31:33.659610 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:33.659577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4" event={"ID":"83275195-3d7e-44ab-b4ef-54edac3cecfc","Type":"ContainerStarted","Data":"83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b"}
Jul 18 02:31:33.659610 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:33.659617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4" event={"ID":"83275195-3d7e-44ab-b4ef-54edac3cecfc","Type":"ContainerStarted","Data":"b1e7ead19d0d5a1202ea1cdb5a79f521a707432d869bc948418b3205b34748ae"}
Jul 18 02:31:33.677340 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:33.677296    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4" podStartSLOduration=1.677282081 podStartE2EDuration="1.677282081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:31:33.675949737 +0000 UTC m=+2586.885663957" watchObservedRunningTime="2026-07-18 02:31:33.677282081 +0000 UTC m=+2586.886996301"
Jul 18 02:31:42.820646 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:42.820615    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:42.820646 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:42.820659    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:42.833086 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:42.833057    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:31:43.720331 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:31:43.720298    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:32:08.692165 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:08.692125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:08.791267 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:08.791230    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:16.542239 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:16.542207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:16.647179 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:16.647144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:20.910074 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:20.910029    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"]
Jul 18 02:32:20.913519 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:20.913502    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:20.916367 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:20.916346    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"default-dockercfg-6fxsl\""
Jul 18 02:32:20.916566 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:20.916548    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"openshift-service-ca.crt\""
Jul 18 02:32:20.916624 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:20.916600    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"kube-root-ca.crt\""
Jul 18 02:32:20.916815 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:20.916793    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0c43e\"/\"rate-test-model-f0c43e-kserve-self-signed-certs\""
Jul 18 02:32:20.922530 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:20.922506    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"]
Jul 18 02:32:20.999642 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:20.999610    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:21.003660 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.003628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tmp-dir\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.003800 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.003667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tls-certs\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.003800 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.003749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-dshm\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.003900 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.003809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-model-cache\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.003900 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.003826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgmkx\" (UniqueName: \"kubernetes.io/projected/4f0c6bb9-3ed9-4056-83c4-d395676b248a-kube-api-access-qgmkx\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.003900 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.003845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-home\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.104668 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.104632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-model-cache\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.104832 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.104736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgmkx\" (UniqueName: \"kubernetes.io/projected/4f0c6bb9-3ed9-4056-83c4-d395676b248a-kube-api-access-qgmkx\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.104832 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.104790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-home\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.104911 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.104831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tmp-dir\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.104911 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.104876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tls-certs\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.105069 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.105028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-dshm\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.105133 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.105088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-model-cache\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.105171 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.105147    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-home\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.105231 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.105210    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tmp-dir\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.107312 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.107284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-dshm\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.107617 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.107600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tls-certs\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.112975 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.112957    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgmkx\" (UniqueName: \"kubernetes.io/projected/4f0c6bb9-3ed9-4056-83c4-d395676b248a-kube-api-access-qgmkx\") pod \"rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") " pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.224098 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.224038    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:21.356351 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.356325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"]
Jul 18 02:32:21.358406 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:32:21.358380    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f0c6bb9_3ed9_4056_83c4_d395676b248a.slice/crio-ecd2aca776c91e44baad54712bdc6970b23fd5fbd0649dc61f911b8a9fb66d6f WatchSource:0}: Error finding container ecd2aca776c91e44baad54712bdc6970b23fd5fbd0649dc61f911b8a9fb66d6f: Status 404 returned error can't find the container with id ecd2aca776c91e44baad54712bdc6970b23fd5fbd0649dc61f911b8a9fb66d6f
Jul 18 02:32:21.902398 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.902355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j" event={"ID":"4f0c6bb9-3ed9-4056-83c4-d395676b248a","Type":"ContainerStarted","Data":"122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334"}
Jul 18 02:32:21.902579 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.902402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j" event={"ID":"4f0c6bb9-3ed9-4056-83c4-d395676b248a","Type":"ContainerStarted","Data":"ecd2aca776c91e44baad54712bdc6970b23fd5fbd0649dc61f911b8a9fb66d6f"}
Jul 18 02:32:21.919400 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:21.919335    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j" podStartSLOduration=1.919317009 podStartE2EDuration="1.919317009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:32:21.919011563 +0000 UTC m=+2635.128725785" watchObservedRunningTime="2026-07-18 02:32:21.919317009 +0000 UTC m=+2635.129031229"
Jul 18 02:32:27.654178 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:27.654147    2580 scope.go:117] "RemoveContainer" containerID="2cfb7c8e1b9d4b4b2f34bdb25dbb3d4a87839bea3d42adc62665c437a3c17a39"
Jul 18 02:32:27.662343 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:27.662325    2580 scope.go:117] "RemoveContainer" containerID="3cbfc3f9325d666a62d4ffceee0f528a588f5f55a8475c3a5e1de66ff5a6f6ff"
Jul 18 02:32:31.224662 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:31.224623    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:31.225089 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:31.224677    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:31.237550 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:31.237528    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:31.967185 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:31.967156    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:32:56.320760 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:56.320719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:56.422432 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:56.422392    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:56.482279 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:32:56.482233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:01.142614 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:01.142579    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"]
Jul 18 02:33:01.165533 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:01.165494    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:02.111269 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:02.111238    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"]
Jul 18 02:33:02.130347 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:02.130307    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:16.163241 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.163166    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4" podUID="83275195-3d7e-44ab-b4ef-54edac3cecfc" containerName="main" containerID="cri-o://83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b" gracePeriod=45
Jul 18 02:33:16.423426 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.423403    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:33:16.511067 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511025    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-dshm\") pod \"83275195-3d7e-44ab-b4ef-54edac3cecfc\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") "
Jul 18 02:33:16.511067 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511070    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83275195-3d7e-44ab-b4ef-54edac3cecfc-tls-certs\") pod \"83275195-3d7e-44ab-b4ef-54edac3cecfc\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") "
Jul 18 02:33:16.511290 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511100    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-home\") pod \"83275195-3d7e-44ab-b4ef-54edac3cecfc\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") "
Jul 18 02:33:16.511290 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511159    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-tmp-dir\") pod \"83275195-3d7e-44ab-b4ef-54edac3cecfc\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") "
Jul 18 02:33:16.511290 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511219    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlms2\" (UniqueName: \"kubernetes.io/projected/83275195-3d7e-44ab-b4ef-54edac3cecfc-kube-api-access-wlms2\") pod \"83275195-3d7e-44ab-b4ef-54edac3cecfc\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") "
Jul 18 02:33:16.511290 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511239    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-model-cache\") pod \"83275195-3d7e-44ab-b4ef-54edac3cecfc\" (UID: \"83275195-3d7e-44ab-b4ef-54edac3cecfc\") "
Jul 18 02:33:16.511508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511387    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-home" (OuterVolumeSpecName: "home") pod "83275195-3d7e-44ab-b4ef-54edac3cecfc" (UID: "83275195-3d7e-44ab-b4ef-54edac3cecfc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:33:16.511508 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511501    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-home\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:16.511605 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511493    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83275195-3d7e-44ab-b4ef-54edac3cecfc" (UID: "83275195-3d7e-44ab-b4ef-54edac3cecfc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:33:16.511643 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.511601    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-model-cache" (OuterVolumeSpecName: "model-cache") pod "83275195-3d7e-44ab-b4ef-54edac3cecfc" (UID: "83275195-3d7e-44ab-b4ef-54edac3cecfc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:33:16.513467 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.513438    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-dshm" (OuterVolumeSpecName: "dshm") pod "83275195-3d7e-44ab-b4ef-54edac3cecfc" (UID: "83275195-3d7e-44ab-b4ef-54edac3cecfc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:33:16.513467 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.513442    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83275195-3d7e-44ab-b4ef-54edac3cecfc-kube-api-access-wlms2" (OuterVolumeSpecName: "kube-api-access-wlms2") pod "83275195-3d7e-44ab-b4ef-54edac3cecfc" (UID: "83275195-3d7e-44ab-b4ef-54edac3cecfc"). InnerVolumeSpecName "kube-api-access-wlms2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:33:16.513467 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.513436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83275195-3d7e-44ab-b4ef-54edac3cecfc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83275195-3d7e-44ab-b4ef-54edac3cecfc" (UID: "83275195-3d7e-44ab-b4ef-54edac3cecfc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:33:16.612549 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.612514    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-dshm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:16.612549 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.612542    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83275195-3d7e-44ab-b4ef-54edac3cecfc-tls-certs\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:16.612762 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.612563    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-tmp-dir\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:16.612762 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.612571    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlms2\" (UniqueName: \"kubernetes.io/projected/83275195-3d7e-44ab-b4ef-54edac3cecfc-kube-api-access-wlms2\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:16.612762 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:16.612580    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83275195-3d7e-44ab-b4ef-54edac3cecfc-model-cache\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:17.132391 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.132345    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j" podUID="4f0c6bb9-3ed9-4056-83c4-d395676b248a" containerName="main" containerID="cri-o://122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334" gracePeriod=45
Jul 18 02:33:17.175740 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.175704    2580 generic.go:356] "Generic (PLEG): container finished" podID="83275195-3d7e-44ab-b4ef-54edac3cecfc" containerID="83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b" exitCode=0
Jul 18 02:33:17.176166 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.175756    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"
Jul 18 02:33:17.176166 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.175794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4" event={"ID":"83275195-3d7e-44ab-b4ef-54edac3cecfc","Type":"ContainerDied","Data":"83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b"}
Jul 18 02:33:17.176166 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.175841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4" event={"ID":"83275195-3d7e-44ab-b4ef-54edac3cecfc","Type":"ContainerDied","Data":"b1e7ead19d0d5a1202ea1cdb5a79f521a707432d869bc948418b3205b34748ae"}
Jul 18 02:33:17.176166 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.175860    2580 scope.go:117] "RemoveContainer" containerID="83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b"
Jul 18 02:33:17.209980 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.209950    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"]
Jul 18 02:33:17.213327 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.213306    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/rate-test-model-d05e31-kserve-7776659d99-ndrm4"]
Jul 18 02:33:17.243704 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.243683    2580 scope.go:117] "RemoveContainer" containerID="83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b"
Jul 18 02:33:17.243993 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:33:17.243969    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b\": container with ID starting with 83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b not found: ID does not exist" containerID="83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b"
Jul 18 02:33:17.244131 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.244000    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b"} err="failed to get container status \"83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b\": rpc error: code = NotFound desc = could not find container \"83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b\": container with ID starting with 83cf061e490593a50462c28f0e23f588225261006f42034520424b59719af68b not found: ID does not exist"
Jul 18 02:33:17.390718 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.390660    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:33:17.391011 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.390989    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83275195-3d7e-44ab-b4ef-54edac3cecfc" path="/var/lib/kubelet/pods/83275195-3d7e-44ab-b4ef-54edac3cecfc/volumes"
Jul 18 02:33:17.520258 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520228    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgmkx\" (UniqueName: \"kubernetes.io/projected/4f0c6bb9-3ed9-4056-83c4-d395676b248a-kube-api-access-qgmkx\") pod \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") "
Jul 18 02:33:17.520450 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520279    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-home\") pod \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") "
Jul 18 02:33:17.520450 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520318    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tls-certs\") pod \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") "
Jul 18 02:33:17.520450 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520343    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tmp-dir\") pod \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") "
Jul 18 02:33:17.520450 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520431    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-dshm\") pod \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") "
Jul 18 02:33:17.520648 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-model-cache\") pod \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\" (UID: \"4f0c6bb9-3ed9-4056-83c4-d395676b248a\") "
Jul 18 02:33:17.520648 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520609    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-home" (OuterVolumeSpecName: "home") pod "4f0c6bb9-3ed9-4056-83c4-d395676b248a" (UID: "4f0c6bb9-3ed9-4056-83c4-d395676b248a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:33:17.520752 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f0c6bb9-3ed9-4056-83c4-d395676b248a" (UID: "4f0c6bb9-3ed9-4056-83c4-d395676b248a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:33:17.520806 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520788    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f0c6bb9-3ed9-4056-83c4-d395676b248a" (UID: "4f0c6bb9-3ed9-4056-83c4-d395676b248a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:33:17.521134 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520967    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-model-cache\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:17.521134 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.520989    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-home\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:17.521134 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.521003    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tmp-dir\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:17.522695 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.522671    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f0c6bb9-3ed9-4056-83c4-d395676b248a" (UID: "4f0c6bb9-3ed9-4056-83c4-d395676b248a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:33:17.523196 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.523170    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f0c6bb9-3ed9-4056-83c4-d395676b248a-kube-api-access-qgmkx" (OuterVolumeSpecName: "kube-api-access-qgmkx") pod "4f0c6bb9-3ed9-4056-83c4-d395676b248a" (UID: "4f0c6bb9-3ed9-4056-83c4-d395676b248a"). InnerVolumeSpecName "kube-api-access-qgmkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:33:17.523296 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.523246    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-dshm" (OuterVolumeSpecName: "dshm") pod "4f0c6bb9-3ed9-4056-83c4-d395676b248a" (UID: "4f0c6bb9-3ed9-4056-83c4-d395676b248a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:33:17.622509 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.622473    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0c6bb9-3ed9-4056-83c4-d395676b248a-dshm\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:17.622509 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.622503    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgmkx\" (UniqueName: \"kubernetes.io/projected/4f0c6bb9-3ed9-4056-83c4-d395676b248a-kube-api-access-qgmkx\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:17.622509 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:17.622514    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0c6bb9-3ed9-4056-83c4-d395676b248a-tls-certs\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:18.182161 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.182131    2580 generic.go:356] "Generic (PLEG): container finished" podID="4f0c6bb9-3ed9-4056-83c4-d395676b248a" containerID="122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334" exitCode=0
Jul 18 02:33:18.182594 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.182176    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"
Jul 18 02:33:18.182594 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.182190    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j" event={"ID":"4f0c6bb9-3ed9-4056-83c4-d395676b248a","Type":"ContainerDied","Data":"122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334"}
Jul 18 02:33:18.182594 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.182220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j" event={"ID":"4f0c6bb9-3ed9-4056-83c4-d395676b248a","Type":"ContainerDied","Data":"ecd2aca776c91e44baad54712bdc6970b23fd5fbd0649dc61f911b8a9fb66d6f"}
Jul 18 02:33:18.182594 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.182236    2580 scope.go:117] "RemoveContainer" containerID="122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334"
Jul 18 02:33:18.191754 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.191731    2580 scope.go:117] "RemoveContainer" containerID="122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334"
Jul 18 02:33:18.191981 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:33:18.191961    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334\": container with ID starting with 122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334 not found: ID does not exist" containerID="122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334"
Jul 18 02:33:18.192025 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.191991    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334"} err="failed to get container status \"122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334\": rpc error: code = NotFound desc = could not find container \"122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334\": container with ID starting with 122015cb5f0ad74490bead66dde85327c469da976dff42e22ff30a0021e2f334 not found: ID does not exist"
Jul 18 02:33:18.202801 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.202779    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"]
Jul 18 02:33:18.208453 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.208434    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f0c43e/rate-test-model-f0c43e-kserve-75b8f875c5-j7p6j"]
Jul 18 02:33:18.446687 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.446606    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7434-140083572479872"]
Jul 18 02:33:18.447038 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.447025    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83275195-3d7e-44ab-b4ef-54edac3cecfc" containerName="main"
Jul 18 02:33:18.447038 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.447053    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="83275195-3d7e-44ab-b4ef-54edac3cecfc" containerName="main"
Jul 18 02:33:18.447156 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.447063    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f0c6bb9-3ed9-4056-83c4-d395676b248a" containerName="main"
Jul 18 02:33:18.447156 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.447069    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f0c6bb9-3ed9-4056-83c4-d395676b248a" containerName="main"
Jul 18 02:33:18.447156 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.447149    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f0c6bb9-3ed9-4056-83c4-d395676b248a" containerName="main"
Jul 18 02:33:18.447276 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.447161    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="83275195-3d7e-44ab-b4ef-54edac3cecfc" containerName="main"
Jul 18 02:33:18.451569 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.451551    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572479872"
Jul 18 02:33:18.454407 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.454383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tkmd9\""
Jul 18 02:33:18.455918 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.455895    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572479872"]
Jul 18 02:33:18.530535 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.530498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk6tv\" (UniqueName: \"kubernetes.io/projected/5cf59e5c-e641-406b-831a-cf4c70fd30f9-kube-api-access-tk6tv\") pod \"test-curl-7434-140083572479872\" (UID: \"5cf59e5c-e641-406b-831a-cf4c70fd30f9\") " pod="opendatahub/test-curl-7434-140083572479872"
Jul 18 02:33:18.631263 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.631231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk6tv\" (UniqueName: \"kubernetes.io/projected/5cf59e5c-e641-406b-831a-cf4c70fd30f9-kube-api-access-tk6tv\") pod \"test-curl-7434-140083572479872\" (UID: \"5cf59e5c-e641-406b-831a-cf4c70fd30f9\") " pod="opendatahub/test-curl-7434-140083572479872"
Jul 18 02:33:18.640002 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.639970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk6tv\" (UniqueName: \"kubernetes.io/projected/5cf59e5c-e641-406b-831a-cf4c70fd30f9-kube-api-access-tk6tv\") pod \"test-curl-7434-140083572479872\" (UID: \"5cf59e5c-e641-406b-831a-cf4c70fd30f9\") " pod="opendatahub/test-curl-7434-140083572479872"
Jul 18 02:33:18.762417 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.762331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572479872"
Jul 18 02:33:18.893082 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.893058    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572479872"]
Jul 18 02:33:18.902504 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:33:18.898239    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cf59e5c_e641_406b_831a_cf4c70fd30f9.slice/crio-77d4a02432662b83065232f63a96a75ff1d42a43a21fc0649165918fd7be7a8e WatchSource:0}: Error finding container 77d4a02432662b83065232f63a96a75ff1d42a43a21fc0649165918fd7be7a8e: Status 404 returned error can't find the container with id 77d4a02432662b83065232f63a96a75ff1d42a43a21fc0649165918fd7be7a8e
Jul 18 02:33:18.902504 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:18.900713    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:33:19.189398 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:19.189363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572479872" event={"ID":"5cf59e5c-e641-406b-831a-cf4c70fd30f9","Type":"ContainerStarted","Data":"77d4a02432662b83065232f63a96a75ff1d42a43a21fc0649165918fd7be7a8e"}
Jul 18 02:33:19.391858 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:19.391823    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f0c6bb9-3ed9-4056-83c4-d395676b248a" path="/var/lib/kubelet/pods/4f0c6bb9-3ed9-4056-83c4-d395676b248a/volumes"
Jul 18 02:33:20.196144 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:20.196106    2580 generic.go:356] "Generic (PLEG): container finished" podID="5cf59e5c-e641-406b-831a-cf4c70fd30f9" containerID="e97c3bd8517ebba0413cb3c96460b4fa9bf168fac357c15f83f05b3f3abb1d1b" exitCode=0
Jul 18 02:33:20.196521 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:20.196195    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572479872" event={"ID":"5cf59e5c-e641-406b-831a-cf4c70fd30f9","Type":"ContainerDied","Data":"e97c3bd8517ebba0413cb3c96460b4fa9bf168fac357c15f83f05b3f3abb1d1b"}
Jul 18 02:33:21.338114 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.338090    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572479872"
Jul 18 02:33:21.357621 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.357593    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7434-140083572479872_5cf59e5c-e641-406b-831a-cf4c70fd30f9/test-curl-7434-140083572479872/0.log"
Jul 18 02:33:21.380711 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.380673    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7434-140083572479872"]
Jul 18 02:33:21.393169 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.393148    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7434-140083572479872"]
Jul 18 02:33:21.457499 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.457428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk6tv\" (UniqueName: \"kubernetes.io/projected/5cf59e5c-e641-406b-831a-cf4c70fd30f9-kube-api-access-tk6tv\") pod \"5cf59e5c-e641-406b-831a-cf4c70fd30f9\" (UID: \"5cf59e5c-e641-406b-831a-cf4c70fd30f9\") "
Jul 18 02:33:21.459576 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.459551    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cf59e5c-e641-406b-831a-cf4c70fd30f9-kube-api-access-tk6tv" (OuterVolumeSpecName: "kube-api-access-tk6tv") pod "5cf59e5c-e641-406b-831a-cf4c70fd30f9" (UID: "5cf59e5c-e641-406b-831a-cf4c70fd30f9"). InnerVolumeSpecName "kube-api-access-tk6tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:33:21.559178 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.559150    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk6tv\" (UniqueName: \"kubernetes.io/projected/5cf59e5c-e641-406b-831a-cf4c70fd30f9-kube-api-access-tk6tv\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:21.624220 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.624192    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"]
Jul 18 02:33:21.624586 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.624574    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cf59e5c-e641-406b-831a-cf4c70fd30f9" containerName="test-curl-7434-140083572479872"
Jul 18 02:33:21.624634 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.624588    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cf59e5c-e641-406b-831a-cf4c70fd30f9" containerName="test-curl-7434-140083572479872"
Jul 18 02:33:21.624670 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.624642    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cf59e5c-e641-406b-831a-cf4c70fd30f9" containerName="test-curl-7434-140083572479872"
Jul 18 02:33:21.627609 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.627592    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"
Jul 18 02:33:21.636100 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.636082    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:33:21.651180 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.651157    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"]
Jul 18 02:33:21.760733 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.760640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf705f4c-af4a-4f19-b776-17ee2141f709-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2\" (UID: \"cf705f4c-af4a-4f19-b776-17ee2141f709\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"
Jul 18 02:33:21.862113 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.862074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf705f4c-af4a-4f19-b776-17ee2141f709-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2\" (UID: \"cf705f4c-af4a-4f19-b776-17ee2141f709\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"
Jul 18 02:33:21.862690 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.862671    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf705f4c-af4a-4f19-b776-17ee2141f709-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2\" (UID: \"cf705f4c-af4a-4f19-b776-17ee2141f709\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"
Jul 18 02:33:21.938095 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:21.938057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"
Jul 18 02:33:22.072300 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:22.072271    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"]
Jul 18 02:33:22.074305 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:33:22.074280    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf705f4c_af4a_4f19_b776_17ee2141f709.slice/crio-2758a9eb69bc74c40184615f9be419da9f9fe53c43cba421c29003663b6b9623 WatchSource:0}: Error finding container 2758a9eb69bc74c40184615f9be419da9f9fe53c43cba421c29003663b6b9623: Status 404 returned error can't find the container with id 2758a9eb69bc74c40184615f9be419da9f9fe53c43cba421c29003663b6b9623
Jul 18 02:33:22.159709 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:22.159682    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:22.208265 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:22.208232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2" event={"ID":"cf705f4c-af4a-4f19-b776-17ee2141f709","Type":"ContainerStarted","Data":"2758a9eb69bc74c40184615f9be419da9f9fe53c43cba421c29003663b6b9623"}
Jul 18 02:33:22.209420 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:22.209401    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572479872"
Jul 18 02:33:22.209538 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:22.209407    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77d4a02432662b83065232f63a96a75ff1d42a43a21fc0649165918fd7be7a8e"
Jul 18 02:33:22.219124 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:33:22.219092    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7434-140083572479872\" is forbidden: User \"system:node:ip-10-0-142-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-56.ec2.internal' and this object" podUID="5cf59e5c-e641-406b-831a-cf4c70fd30f9" pod="opendatahub/test-curl-7434-140083572479872"
Jul 18 02:33:22.257156 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:22.257124    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:23.216015 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:23.215981    2580 generic.go:356] "Generic (PLEG): container finished" podID="cf705f4c-af4a-4f19-b776-17ee2141f709" containerID="43e14086fdb9dda04dccf0e924b15df40a93064aba92d9cfadd18848c3faabf1" exitCode=0
Jul 18 02:33:23.216448 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:23.216080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2" event={"ID":"cf705f4c-af4a-4f19-b776-17ee2141f709","Type":"ContainerDied","Data":"43e14086fdb9dda04dccf0e924b15df40a93064aba92d9cfadd18848c3faabf1"}
Jul 18 02:33:23.218273 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:33:23.218249    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7434-140083572479872\" is forbidden: User \"system:node:ip-10-0-142-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-56.ec2.internal' and this object" podUID="5cf59e5c-e641-406b-831a-cf4c70fd30f9" pod="opendatahub/test-curl-7434-140083572479872"
Jul 18 02:33:23.391296 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:23.391267    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cf59e5c-e641-406b-831a-cf4c70fd30f9" path="/var/lib/kubelet/pods/5cf59e5c-e641-406b-831a-cf4c70fd30f9/volumes"
Jul 18 02:33:24.347960 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:24.347937    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"
Jul 18 02:33:24.483786 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:24.483711    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf705f4c-af4a-4f19-b776-17ee2141f709-maas-api-service-ca\") pod \"cf705f4c-af4a-4f19-b776-17ee2141f709\" (UID: \"cf705f4c-af4a-4f19-b776-17ee2141f709\") "
Jul 18 02:33:24.484084 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:24.484038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf705f4c-af4a-4f19-b776-17ee2141f709-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf705f4c-af4a-4f19-b776-17ee2141f709" (UID: "cf705f4c-af4a-4f19-b776-17ee2141f709"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:33:24.584542 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:24.584513    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf705f4c-af4a-4f19-b776-17ee2141f709-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:25.226346 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:25.226314    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"
Jul 18 02:33:25.226509 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:25.226357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2" event={"ID":"cf705f4c-af4a-4f19-b776-17ee2141f709","Type":"ContainerDied","Data":"2758a9eb69bc74c40184615f9be419da9f9fe53c43cba421c29003663b6b9623"}
Jul 18 02:33:25.226509 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:25.226412    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2758a9eb69bc74c40184615f9be419da9f9fe53c43cba421c29003663b6b9623"
Jul 18 02:33:32.108715 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.108680    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"]
Jul 18 02:33:32.109269 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.108917    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4" podUID="75b815e4-aef6-416a-a7ce-e787c9de5b25" containerName="maas-api" containerID="cri-o://87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68" gracePeriod=30
Jul 18 02:33:32.142643 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.142610    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:32.157135 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.157109    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm"]
Jul 18 02:33:32.159829 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.159810    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d05e31-29739030-l78vm"]
Jul 18 02:33:32.564849 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.564822    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:33:32.657614 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.657522    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b815e4-aef6-416a-a7ce-e787c9de5b25-maas-api-tls\") pod \"75b815e4-aef6-416a-a7ce-e787c9de5b25\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") "
Jul 18 02:33:32.657796 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.657614    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7vp2\" (UniqueName: \"kubernetes.io/projected/75b815e4-aef6-416a-a7ce-e787c9de5b25-kube-api-access-s7vp2\") pod \"75b815e4-aef6-416a-a7ce-e787c9de5b25\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") "
Jul 18 02:33:32.657796 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.657668    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b815e4-aef6-416a-a7ce-e787c9de5b25-openshift-service-ca\") pod \"75b815e4-aef6-416a-a7ce-e787c9de5b25\" (UID: \"75b815e4-aef6-416a-a7ce-e787c9de5b25\") "
Jul 18 02:33:32.658033 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.658010    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b815e4-aef6-416a-a7ce-e787c9de5b25-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "75b815e4-aef6-416a-a7ce-e787c9de5b25" (UID: "75b815e4-aef6-416a-a7ce-e787c9de5b25"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:33:32.659779 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.659758    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b815e4-aef6-416a-a7ce-e787c9de5b25-kube-api-access-s7vp2" (OuterVolumeSpecName: "kube-api-access-s7vp2") pod "75b815e4-aef6-416a-a7ce-e787c9de5b25" (UID: "75b815e4-aef6-416a-a7ce-e787c9de5b25"). InnerVolumeSpecName "kube-api-access-s7vp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:33:32.659844 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.659831    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b815e4-aef6-416a-a7ce-e787c9de5b25-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "75b815e4-aef6-416a-a7ce-e787c9de5b25" (UID: "75b815e4-aef6-416a-a7ce-e787c9de5b25"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:33:32.758612 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.758577    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7vp2\" (UniqueName: \"kubernetes.io/projected/75b815e4-aef6-416a-a7ce-e787c9de5b25-kube-api-access-s7vp2\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:32.758612 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.758608    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b815e4-aef6-416a-a7ce-e787c9de5b25-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:32.758612 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:32.758618    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b815e4-aef6-416a-a7ce-e787c9de5b25-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:33.267214 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.267183    2580 generic.go:356] "Generic (PLEG): container finished" podID="75b815e4-aef6-416a-a7ce-e787c9de5b25" containerID="87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68" exitCode=0
Jul 18 02:33:33.267630 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.267234    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"
Jul 18 02:33:33.267630 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.267274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4" event={"ID":"75b815e4-aef6-416a-a7ce-e787c9de5b25","Type":"ContainerDied","Data":"87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68"}
Jul 18 02:33:33.267630 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.267330    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4" event={"ID":"75b815e4-aef6-416a-a7ce-e787c9de5b25","Type":"ContainerDied","Data":"b45bbaa12628d19f67892964c2b485c284e2715bfc28eb1195d4fc05e3835155"}
Jul 18 02:33:33.267630 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.267359    2580 scope.go:117] "RemoveContainer" containerID="87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68"
Jul 18 02:33:33.279113 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.279038    2580 scope.go:117] "RemoveContainer" containerID="87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68"
Jul 18 02:33:33.279332 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:33:33.279308    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68\": container with ID starting with 87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68 not found: ID does not exist" containerID="87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68"
Jul 18 02:33:33.279430 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.279337    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68"} err="failed to get container status \"87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68\": rpc error: code = NotFound desc = could not find container \"87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68\": container with ID starting with 87c28880bc113f11e1be1a7e98d310357e8c26ff63e6c4217ecf7bdd985dea68 not found: ID does not exist"
Jul 18 02:33:33.287799 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.287780    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"]
Jul 18 02:33:33.291404 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.291384    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d05e31-548fb9c6b4-c5mh4"]
Jul 18 02:33:33.390233 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.390204    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75b815e4-aef6-416a-a7ce-e787c9de5b25" path="/var/lib/kubelet/pods/75b815e4-aef6-416a-a7ce-e787c9de5b25/volumes"
Jul 18 02:33:33.390496 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:33.390483    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0b1c46c-a0e7-4d73-b544-f1051409df12" path="/var/lib/kubelet/pods/e0b1c46c-a0e7-4d73-b544-f1051409df12/volumes"
Jul 18 02:33:36.668498 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:36.668463    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"]
Jul 18 02:33:36.672756 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:36.672732    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d05e31-cd5d2"]
Jul 18 02:33:36.706634 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:36.706595    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:37.098656 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:37.098568    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:37.390471 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:37.390379    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf705f4c-af4a-4f19-b776-17ee2141f709" path="/var/lib/kubelet/pods/cf705f4c-af4a-4f19-b776-17ee2141f709/volumes"
Jul 18 02:33:43.215769 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.215735    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:43.278303 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.278270    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:43.465027 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.464989    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:43.693947 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.693916    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:43.808664 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.808632    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"]
Jul 18 02:33:43.809077 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.809061    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75b815e4-aef6-416a-a7ce-e787c9de5b25" containerName="maas-api"
Jul 18 02:33:43.809182 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.809080    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b815e4-aef6-416a-a7ce-e787c9de5b25" containerName="maas-api"
Jul 18 02:33:43.809182 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.809100    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf705f4c-af4a-4f19-b776-17ee2141f709" containerName="revoke-keys"
Jul 18 02:33:43.809182 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.809108    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf705f4c-af4a-4f19-b776-17ee2141f709" containerName="revoke-keys"
Jul 18 02:33:43.809341 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.809215    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf705f4c-af4a-4f19-b776-17ee2141f709" containerName="revoke-keys"
Jul 18 02:33:43.809341 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.809232    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="75b815e4-aef6-416a-a7ce-e787c9de5b25" containerName="maas-api"
Jul 18 02:33:43.812352 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.812332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"
Jul 18 02:33:43.815017 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.814991    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\""
Jul 18 02:33:43.819900 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.819876    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"]
Jul 18 02:33:43.853349 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.853318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea495c8-8d02-43be-86d4-b0f8fbf365ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t\" (UID: \"1ea495c8-8d02-43be-86d4-b0f8fbf365ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"
Jul 18 02:33:43.954029 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.953954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea495c8-8d02-43be-86d4-b0f8fbf365ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t\" (UID: \"1ea495c8-8d02-43be-86d4-b0f8fbf365ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"
Jul 18 02:33:43.954594 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:43.954574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea495c8-8d02-43be-86d4-b0f8fbf365ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t\" (UID: \"1ea495c8-8d02-43be-86d4-b0f8fbf365ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"
Jul 18 02:33:44.123510 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:44.123476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"
Jul 18 02:33:44.256257 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:44.256232    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"]
Jul 18 02:33:44.259034 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:33:44.259009    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea495c8_8d02_43be_86d4_b0f8fbf365ae.slice/crio-b179b181c0c8035b81750b98a22f0566ef72d4460ef9284c47381aa664470ea0 WatchSource:0}: Error finding container b179b181c0c8035b81750b98a22f0566ef72d4460ef9284c47381aa664470ea0: Status 404 returned error can't find the container with id b179b181c0c8035b81750b98a22f0566ef72d4460ef9284c47381aa664470ea0
Jul 18 02:33:44.303232 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:44.303203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:44.327661 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:44.327627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t" event={"ID":"1ea495c8-8d02-43be-86d4-b0f8fbf365ae","Type":"ContainerStarted","Data":"b179b181c0c8035b81750b98a22f0566ef72d4460ef9284c47381aa664470ea0"}
Jul 18 02:33:44.404360 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:44.404322    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:45.333906 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:45.333871    2580 generic.go:356] "Generic (PLEG): container finished" podID="1ea495c8-8d02-43be-86d4-b0f8fbf365ae" containerID="4c8205dba6f1990f00e3da9510860336be88b252dd5e05cdffb6cb247ac147c8" exitCode=0
Jul 18 02:33:45.333906 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:45.333907    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t" event={"ID":"1ea495c8-8d02-43be-86d4-b0f8fbf365ae","Type":"ContainerDied","Data":"4c8205dba6f1990f00e3da9510860336be88b252dd5e05cdffb6cb247ac147c8"}
Jul 18 02:33:46.472907 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:46.472884    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"
Jul 18 02:33:46.577273 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:46.577108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea495c8-8d02-43be-86d4-b0f8fbf365ae-maas-api-service-ca\") pod \"1ea495c8-8d02-43be-86d4-b0f8fbf365ae\" (UID: \"1ea495c8-8d02-43be-86d4-b0f8fbf365ae\") "
Jul 18 02:33:46.577791 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:46.577758    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ea495c8-8d02-43be-86d4-b0f8fbf365ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1ea495c8-8d02-43be-86d4-b0f8fbf365ae" (UID: "1ea495c8-8d02-43be-86d4-b0f8fbf365ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:33:46.578680 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:46.578655    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea495c8-8d02-43be-86d4-b0f8fbf365ae-maas-api-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:47.345750 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:47.345714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t" event={"ID":"1ea495c8-8d02-43be-86d4-b0f8fbf365ae","Type":"ContainerDied","Data":"b179b181c0c8035b81750b98a22f0566ef72d4460ef9284c47381aa664470ea0"}
Jul 18 02:33:47.345937 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:47.345790    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b179b181c0c8035b81750b98a22f0566ef72d4460ef9284c47381aa664470ea0"
Jul 18 02:33:47.345937 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:47.345729    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"
Jul 18 02:33:54.163165 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.163133    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"]
Jul 18 02:33:54.163601 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.163355    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7" podUID="49d1c081-a7d8-4b0d-ac31-2039edccfb34" containerName="maas-api" containerID="cri-o://b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e" gracePeriod=30
Jul 18 02:33:54.188295 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.188268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:54.192087 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.192064    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl"]
Jul 18 02:33:54.194577 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.194558    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f0c43e-29739030-slcnl"]
Jul 18 02:33:54.625893 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.625868    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:33:54.754851 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.754766    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d1c081-a7d8-4b0d-ac31-2039edccfb34-maas-api-tls\") pod \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") "
Jul 18 02:33:54.754851 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.754826    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj9qr\" (UniqueName: \"kubernetes.io/projected/49d1c081-a7d8-4b0d-ac31-2039edccfb34-kube-api-access-mj9qr\") pod \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") "
Jul 18 02:33:54.755065 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.754862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d1c081-a7d8-4b0d-ac31-2039edccfb34-openshift-service-ca\") pod \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\" (UID: \"49d1c081-a7d8-4b0d-ac31-2039edccfb34\") "
Jul 18 02:33:54.755322 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.755288    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49d1c081-a7d8-4b0d-ac31-2039edccfb34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "49d1c081-a7d8-4b0d-ac31-2039edccfb34" (UID: "49d1c081-a7d8-4b0d-ac31-2039edccfb34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:33:54.757071 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.757030    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d1c081-a7d8-4b0d-ac31-2039edccfb34-kube-api-access-mj9qr" (OuterVolumeSpecName: "kube-api-access-mj9qr") pod "49d1c081-a7d8-4b0d-ac31-2039edccfb34" (UID: "49d1c081-a7d8-4b0d-ac31-2039edccfb34"). InnerVolumeSpecName "kube-api-access-mj9qr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:33:54.757163 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.757123    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49d1c081-a7d8-4b0d-ac31-2039edccfb34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "49d1c081-a7d8-4b0d-ac31-2039edccfb34" (UID: "49d1c081-a7d8-4b0d-ac31-2039edccfb34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:33:54.856293 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.856269    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d1c081-a7d8-4b0d-ac31-2039edccfb34-maas-api-tls\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:54.856293 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.856293    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj9qr\" (UniqueName: \"kubernetes.io/projected/49d1c081-a7d8-4b0d-ac31-2039edccfb34-kube-api-access-mj9qr\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:54.856452 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:54.856302    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d1c081-a7d8-4b0d-ac31-2039edccfb34-openshift-service-ca\") on node \"ip-10-0-142-56.ec2.internal\" DevicePath \"\""
Jul 18 02:33:55.391239 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.391203    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30eab2ad-2643-487b-80ed-d6e868803ad7" path="/var/lib/kubelet/pods/30eab2ad-2643-487b-80ed-d6e868803ad7/volumes"
Jul 18 02:33:55.392495 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.392461    2580 generic.go:356] "Generic (PLEG): container finished" podID="49d1c081-a7d8-4b0d-ac31-2039edccfb34" containerID="b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e" exitCode=0
Jul 18 02:33:55.392632 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.392500    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"
Jul 18 02:33:55.392632 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.392520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7" event={"ID":"49d1c081-a7d8-4b0d-ac31-2039edccfb34","Type":"ContainerDied","Data":"b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e"}
Jul 18 02:33:55.392632 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.392582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7" event={"ID":"49d1c081-a7d8-4b0d-ac31-2039edccfb34","Type":"ContainerDied","Data":"6e08472d6fe17447365769d4fe61bdf0864505f5ecb16d3596070af5bbb12f0b"}
Jul 18 02:33:55.392632 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.392614    2580 scope.go:117] "RemoveContainer" containerID="b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e"
Jul 18 02:33:55.404143 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.404119    2580 scope.go:117] "RemoveContainer" containerID="b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e"
Jul 18 02:33:55.404769 ip-10-0-142-56 kubenswrapper[2580]: E0718 02:33:55.404748    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e\": container with ID starting with b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e not found: ID does not exist" containerID="b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e"
Jul 18 02:33:55.404861 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.404776    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e"} err="failed to get container status \"b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e\": rpc error: code = NotFound desc = could not find container \"b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e\": container with ID starting with b8569888442085cceddd210b2fc8827504482e55998bf5e09ba18d5be594948e not found: ID does not exist"
Jul 18 02:33:55.417714 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.417684    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"]
Jul 18 02:33:55.420878 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:55.420857    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f0c43e-789d76ff8d-g4gq7"]
Jul 18 02:33:57.390747 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:57.390707    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49d1c081-a7d8-4b0d-ac31-2039edccfb34" path="/var/lib/kubelet/pods/49d1c081-a7d8-4b0d-ac31-2039edccfb34/volumes"
Jul 18 02:33:58.868321 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:58.868286    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"]
Jul 18 02:33:58.869756 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:58.869737    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0c43e-h2n8t"]
Jul 18 02:33:58.872483 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:58.872454    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:59.230861 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:59.230817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:59.391313 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:33:59.391283    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ea495c8-8d02-43be-86d4-b0f8fbf365ae" path="/var/lib/kubelet/pods/1ea495c8-8d02-43be-86d4-b0f8fbf365ae/volumes"
Jul 18 02:34:05.089501 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:05.089465    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:34:05.147227 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:05.147193    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:34:05.337974 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:05.337931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:34:05.559191 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:05.559153    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:34:11.931385 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:11.931359    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-lbflf_d0b35551-ad37-49f0-aa8f-f94340c1828e/copy-manifests/0.log"
Jul 18 02:34:11.937954 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:11.937924    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-lbflf_d0b35551-ad37-49f0-aa8f-f94340c1828e/manager/0.log"
Jul 18 02:34:14.693582 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:14.693545    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gwqr7_79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2/manager/0.log"
Jul 18 02:34:15.355887 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:15.355857    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c65d4b6f-nwxt2_6e9d01ad-b24f-4469-852c-50b8f78ef8df/kube-auth-proxy/0.log"
Jul 18 02:34:15.581539 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:15.581492    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wg7gq_ed1d004c-b99f-4c1c-b665-86e51c2d53fb/payload-pre-processing/0.log"
Jul 18 02:34:16.231196 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:16.231152    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5m4pz_301922eb-9209-4660-b46d-47361cc5c326/main/0.log"
Jul 18 02:34:16.454272 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:16.454218    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbdpsc_5f3a31b2-f2f8-4004-8ba9-aea20c69c193/main/0.log"
Jul 18 02:34:21.621894 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.621858    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-28rtf/must-gather-vhgww"]
Jul 18 02:34:21.622327 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.622257    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ea495c8-8d02-43be-86d4-b0f8fbf365ae" containerName="revoke-keys"
Jul 18 02:34:21.622327 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.622269    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ea495c8-8d02-43be-86d4-b0f8fbf365ae" containerName="revoke-keys"
Jul 18 02:34:21.622327 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.622278    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d1c081-a7d8-4b0d-ac31-2039edccfb34" containerName="maas-api"
Jul 18 02:34:21.622327 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.622283    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d1c081-a7d8-4b0d-ac31-2039edccfb34" containerName="maas-api"
Jul 18 02:34:21.622468 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.622333    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d1c081-a7d8-4b0d-ac31-2039edccfb34" containerName="maas-api"
Jul 18 02:34:21.622468 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.622341    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ea495c8-8d02-43be-86d4-b0f8fbf365ae" containerName="revoke-keys"
Jul 18 02:34:21.631710 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.631682    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-28rtf/must-gather-vhgww"
Jul 18 02:34:21.634673 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.634648    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-28rtf\"/\"openshift-service-ca.crt\""
Jul 18 02:34:21.635337 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.635314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-28rtf/must-gather-vhgww"]
Jul 18 02:34:21.635578 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.635562    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-28rtf\"/\"default-dockercfg-5hl2v\""
Jul 18 02:34:21.635659 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.635620    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-28rtf\"/\"kube-root-ca.crt\""
Jul 18 02:34:21.637536 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.637509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:34:21.686258 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.686237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4112318b-0252-492c-96d8-4c51bebd747c-must-gather-output\") pod \"must-gather-vhgww\" (UID: \"4112318b-0252-492c-96d8-4c51bebd747c\") " pod="openshift-must-gather-28rtf/must-gather-vhgww"
Jul 18 02:34:21.686379 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.686265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpfdw\" (UniqueName: \"kubernetes.io/projected/4112318b-0252-492c-96d8-4c51bebd747c-kube-api-access-dpfdw\") pod \"must-gather-vhgww\" (UID: \"4112318b-0252-492c-96d8-4c51bebd747c\") " pod="openshift-must-gather-28rtf/must-gather-vhgww"
Jul 18 02:34:21.741417 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.741385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:34:21.786794 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.786764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4112318b-0252-492c-96d8-4c51bebd747c-must-gather-output\") pod \"must-gather-vhgww\" (UID: \"4112318b-0252-492c-96d8-4c51bebd747c\") " pod="openshift-must-gather-28rtf/must-gather-vhgww"
Jul 18 02:34:21.786928 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.786802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfdw\" (UniqueName: \"kubernetes.io/projected/4112318b-0252-492c-96d8-4c51bebd747c-kube-api-access-dpfdw\") pod \"must-gather-vhgww\" (UID: \"4112318b-0252-492c-96d8-4c51bebd747c\") " pod="openshift-must-gather-28rtf/must-gather-vhgww"
Jul 18 02:34:21.787189 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.787160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4112318b-0252-492c-96d8-4c51bebd747c-must-gather-output\") pod \"must-gather-vhgww\" (UID: \"4112318b-0252-492c-96d8-4c51bebd747c\") " pod="openshift-must-gather-28rtf/must-gather-vhgww"
Jul 18 02:34:21.795763 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.795741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpfdw\" (UniqueName: \"kubernetes.io/projected/4112318b-0252-492c-96d8-4c51bebd747c-kube-api-access-dpfdw\") pod \"must-gather-vhgww\" (UID: \"4112318b-0252-492c-96d8-4c51bebd747c\") " pod="openshift-must-gather-28rtf/must-gather-vhgww"
Jul 18 02:34:21.942312 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:21.942287    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-28rtf/must-gather-vhgww"
Jul 18 02:34:22.075491 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:22.075457    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-28rtf/must-gather-vhgww"]
Jul 18 02:34:22.078615 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:34:22.078587    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4112318b_0252_492c_96d8_4c51bebd747c.slice/crio-36bb507951212189e6e625138941ad602c22eff8f448eec8f3d792b467a84df0 WatchSource:0}: Error finding container 36bb507951212189e6e625138941ad602c22eff8f448eec8f3d792b467a84df0: Status 404 returned error can't find the container with id 36bb507951212189e6e625138941ad602c22eff8f448eec8f3d792b467a84df0
Jul 18 02:34:22.524325 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:22.524288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-28rtf/must-gather-vhgww" event={"ID":"4112318b-0252-492c-96d8-4c51bebd747c","Type":"ContainerStarted","Data":"36bb507951212189e6e625138941ad602c22eff8f448eec8f3d792b467a84df0"}
Jul 18 02:34:23.533288 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:23.533244    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-28rtf/must-gather-vhgww" event={"ID":"4112318b-0252-492c-96d8-4c51bebd747c","Type":"ContainerStarted","Data":"8150dd3d56870e90ce6514a2da5a6af8c80733ce17de4802d2c7d0a56bb22dbd"}
Jul 18 02:34:23.533778 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:23.533304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-28rtf/must-gather-vhgww" event={"ID":"4112318b-0252-492c-96d8-4c51bebd747c","Type":"ContainerStarted","Data":"c77dbe9501fc03b347f5064dec03d26dd1519372f875e67620bbf04efd879a98"}
Jul 18 02:34:23.550778 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:23.550716    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-28rtf/must-gather-vhgww" podStartSLOduration=1.763995046 podStartE2EDuration="2.550700015s" totalImagesPullingTime="786.704969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:34:23.548528743 +0000 UTC m=+2756.758242964" watchObservedRunningTime="2026-07-18 02:34:23.550700015 +0000 UTC m=+2756.760414236"
Jul 18 02:34:24.424690 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:24.424647    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qj8km_42e00b16-b4d4-4e92-8a85-b4477120c978/global-pull-secret-syncer/0.log"
Jul 18 02:34:24.530521 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:24.530490    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log"
Jul 18 02:34:24.666002 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:24.665972    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-56.ec2.internal_264b181dc92a9549d9a55f9f442fc196/haproxy/0.log"
Jul 18 02:34:30.621687 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:30.620019    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gwqr7_79e8a9b7-fe08-4d3e-9bf6-19bd688a17c2/manager/0.log"
Jul 18 02:34:31.691835 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:31.691798    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9gn8_3bc46aff-98b6-4d1d-81eb-eaa4022de5b2/konnectivity-agent/0.log" limit=1024
Jul 18 02:34:32.414580 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.414552    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_991e70a6-ba90-465e-88a4-2fa8c000fe77/alertmanager/0.log"
Jul 18 02:34:32.436819 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.436794    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_991e70a6-ba90-465e-88a4-2fa8c000fe77/config-reloader/0.log"
Jul 18 02:34:32.456394 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.456308    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_991e70a6-ba90-465e-88a4-2fa8c000fe77/kube-rbac-proxy-web/0.log"
Jul 18 02:34:32.476199 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.476173    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_991e70a6-ba90-465e-88a4-2fa8c000fe77/kube-rbac-proxy/0.log"
Jul 18 02:34:32.501900 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.501868    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_991e70a6-ba90-465e-88a4-2fa8c000fe77/kube-rbac-proxy-metric/0.log"
Jul 18 02:34:32.522167 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.522142    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_991e70a6-ba90-465e-88a4-2fa8c000fe77/prom-label-proxy/0.log"
Jul 18 02:34:32.547702 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.547676    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_991e70a6-ba90-465e-88a4-2fa8c000fe77/init-config-reloader/0.log"
Jul 18 02:34:32.627489 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.627402    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-vhx57_dc360c2e-bd30-45af-b4c9-9d1e280dfdf0/kube-state-metrics/0.log"
Jul 18 02:34:32.643248 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.643168    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-vhx57_dc360c2e-bd30-45af-b4c9-9d1e280dfdf0/kube-rbac-proxy-main/0.log"
Jul 18 02:34:32.661735 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.661705    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-vhx57_dc360c2e-bd30-45af-b4c9-9d1e280dfdf0/kube-rbac-proxy-self/0.log"
Jul 18 02:34:32.704728 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.704694    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6564b44b7-4btf9_8543b4d1-c093-45d1-8df8-4a0be6775d93/metrics-server/0.log"
Jul 18 02:34:32.840352 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.840277    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rg7pk_1b769a28-9d0a-4189-9f5d-17eb826d0e8e/node-exporter/0.log"
Jul 18 02:34:32.862414 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.862386    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rg7pk_1b769a28-9d0a-4189-9f5d-17eb826d0e8e/kube-rbac-proxy/0.log"
Jul 18 02:34:32.882986 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:32.882962    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rg7pk_1b769a28-9d0a-4189-9f5d-17eb826d0e8e/init-textfile/0.log"
Jul 18 02:34:33.082605 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.082581    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_775e83c2-ae5f-4389-8198-55ed9a7921ae/prometheus/0.log"
Jul 18 02:34:33.103542 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.103449    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_775e83c2-ae5f-4389-8198-55ed9a7921ae/config-reloader/0.log"
Jul 18 02:34:33.124893 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.124866    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_775e83c2-ae5f-4389-8198-55ed9a7921ae/thanos-sidecar/0.log"
Jul 18 02:34:33.150904 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.150877    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_775e83c2-ae5f-4389-8198-55ed9a7921ae/kube-rbac-proxy-web/0.log"
Jul 18 02:34:33.166254 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.166225    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_775e83c2-ae5f-4389-8198-55ed9a7921ae/kube-rbac-proxy/0.log"
Jul 18 02:34:33.185460 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.185425    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_775e83c2-ae5f-4389-8198-55ed9a7921ae/kube-rbac-proxy-thanos/0.log"
Jul 18 02:34:33.205305 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.205276    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_775e83c2-ae5f-4389-8198-55ed9a7921ae/init-config-reloader/0.log"
Jul 18 02:34:33.235867 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.235834    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-vf87r_719da8da-43ff-4692-ad70-b84987e714ce/prometheus-operator/0.log"
Jul 18 02:34:33.250166 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.250138    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-vf87r_719da8da-43ff-4692-ad70-b84987e714ce/kube-rbac-proxy/0.log"
Jul 18 02:34:33.270687 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.270660    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-dcxqz_7545c793-d408-4399-bd1b-adcf7ceddc7f/prometheus-operator-admission-webhook/0.log"
Jul 18 02:34:33.366731 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.366658    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69dff55877-z9vlg_f8ab39c5-194a-483c-98b2-2ab6a4f987f4/thanos-query/0.log"
Jul 18 02:34:33.390746 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.390718    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69dff55877-z9vlg_f8ab39c5-194a-483c-98b2-2ab6a4f987f4/kube-rbac-proxy-web/0.log"
Jul 18 02:34:33.413107 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.413005    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69dff55877-z9vlg_f8ab39c5-194a-483c-98b2-2ab6a4f987f4/kube-rbac-proxy/0.log"
Jul 18 02:34:33.432580 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.432552    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69dff55877-z9vlg_f8ab39c5-194a-483c-98b2-2ab6a4f987f4/prom-label-proxy/0.log"
Jul 18 02:34:33.451843 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.451804    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69dff55877-z9vlg_f8ab39c5-194a-483c-98b2-2ab6a4f987f4/kube-rbac-proxy-rules/0.log"
Jul 18 02:34:33.475496 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:33.475463    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69dff55877-z9vlg_f8ab39c5-194a-483c-98b2-2ab6a4f987f4/kube-rbac-proxy-metrics/0.log"
Jul 18 02:34:34.343344 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.343306    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"]
Jul 18 02:34:34.350544 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.350513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.358723 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.358697    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"]
Jul 18 02:34:34.422068 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.421220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq4t6\" (UniqueName: \"kubernetes.io/projected/d029e2eb-736f-45cb-aca5-649101fcb51c-kube-api-access-nq4t6\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.422068 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.421273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-proc\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.422068 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.421303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-lib-modules\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.422068 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.421479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-sys\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.422068 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.421511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-podres\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.522716 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.522661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-sys\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.522716 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.522718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-podres\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.523030 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.522763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq4t6\" (UniqueName: \"kubernetes.io/projected/d029e2eb-736f-45cb-aca5-649101fcb51c-kube-api-access-nq4t6\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.523030 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.522773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-sys\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.523030 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.522797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-proc\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.523030 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.522854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-lib-modules\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.523030 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.522865    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-proc\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.523030 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.522910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-podres\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.523402 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.523079    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d029e2eb-736f-45cb-aca5-649101fcb51c-lib-modules\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.531765 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.531732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq4t6\" (UniqueName: \"kubernetes.io/projected/d029e2eb-736f-45cb-aca5-649101fcb51c-kube-api-access-nq4t6\") pod \"perf-node-gather-daemonset-9ztqw\" (UID: \"d029e2eb-736f-45cb-aca5-649101fcb51c\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.667647 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.667580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:34.854220 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:34.853912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"]
Jul 18 02:34:34.858271 ip-10-0-142-56 kubenswrapper[2580]: W0718 02:34:34.858228    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd029e2eb_736f_45cb_aca5_649101fcb51c.slice/crio-45e580f202f8bea63cae8c144a19f563dd2b507785e0076b68ac0af9a97a6bec WatchSource:0}: Error finding container 45e580f202f8bea63cae8c144a19f563dd2b507785e0076b68ac0af9a97a6bec: Status 404 returned error can't find the container with id 45e580f202f8bea63cae8c144a19f563dd2b507785e0076b68ac0af9a97a6bec
Jul 18 02:34:35.643753 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:35.643709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw" event={"ID":"d029e2eb-736f-45cb-aca5-649101fcb51c","Type":"ContainerStarted","Data":"1c40bc6b9b3fe8056bb5e3e773038302b2de0b7105b41aeb1ed21daa178e8ce8"}
Jul 18 02:34:35.644251 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:35.643776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw" event={"ID":"d029e2eb-736f-45cb-aca5-649101fcb51c","Type":"ContainerStarted","Data":"45e580f202f8bea63cae8c144a19f563dd2b507785e0076b68ac0af9a97a6bec"}
Jul 18 02:34:35.644251 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:35.644102    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:35.662595 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:35.662534    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw" podStartSLOduration=1.662512715 podStartE2EDuration="1.662512715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:34:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:34:35.660893797 +0000 UTC m=+2768.870608022" watchObservedRunningTime="2026-07-18 02:34:35.662512715 +0000 UTC m=+2768.872226939"
Jul 18 02:34:37.374032 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:37.373992    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xrhq8_acfc916f-b1f4-4271-a49e-b20783422922/dns/0.log"
Jul 18 02:34:37.392998 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:37.392972    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xrhq8_acfc916f-b1f4-4271-a49e-b20783422922/kube-rbac-proxy/0.log"
Jul 18 02:34:37.505468 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:37.505440    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jk4xm_8bc9c16a-612f-4d70-a673-79af0c13a275/dns-node-resolver/0.log"
Jul 18 02:34:38.098379 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:38.098351    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lpmsm_108086ce-a8d1-46d1-904d-86c21e823ad7/node-ca/0.log"
Jul 18 02:34:39.156282 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:39.156251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c65d4b6f-nwxt2_6e9d01ad-b24f-4469-852c-50b8f78ef8df/kube-auth-proxy/0.log"
Jul 18 02:34:39.257291 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:39.257240    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wg7gq_ed1d004c-b99f-4c1c-b665-86e51c2d53fb/payload-pre-processing/0.log"
Jul 18 02:34:39.997320 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:39.997281    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xz6vx_892559b2-c9ea-43d1-8f23-0e41d98f63a3/serve-healthcheck-canary/0.log"
Jul 18 02:34:40.541406 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:40.541367    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qgmwb_dad73e0c-2967-4b12-8d9f-c45337cedb3f/kube-rbac-proxy/0.log"
Jul 18 02:34:40.562581 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:40.562549    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qgmwb_dad73e0c-2967-4b12-8d9f-c45337cedb3f/exporter/0.log"
Jul 18 02:34:40.582902 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:40.582867    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qgmwb_dad73e0c-2967-4b12-8d9f-c45337cedb3f/extractor/0.log"
Jul 18 02:34:41.661495 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:41.661461    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-9ztqw"
Jul 18 02:34:43.277881 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:43.277814    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-lbflf_d0b35551-ad37-49f0-aa8f-f94340c1828e/manager/0.log"
Jul 18 02:34:43.299652 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:43.299578    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-lbflf_d0b35551-ad37-49f0-aa8f-f94340c1828e/copy-manifests/0.log"
Jul 18 02:34:44.719814 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:44.719774    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6bc78dbd6c-fpqjb_0b9531fb-63b3-44b5-ac95-2e5e431ac240/manager/0.log"
Jul 18 02:34:45.602721 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:45.602679    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c65d4b6f-nwxt2_6e9d01ad-b24f-4469-852c-50b8f78ef8df/kube-auth-proxy/0.log"
Jul 18 02:34:45.689440 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:45.689411    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wg7gq_ed1d004c-b99f-4c1c-b665-86e51c2d53fb/payload-pre-processing/0.log"
Jul 18 02:34:47.023718 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:47.023685    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c65d4b6f-nwxt2_6e9d01ad-b24f-4469-852c-50b8f78ef8df/kube-auth-proxy/0.log"
Jul 18 02:34:47.109335 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:47.109302    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wg7gq_ed1d004c-b99f-4c1c-b665-86e51c2d53fb/payload-pre-processing/0.log"
Jul 18 02:34:51.118608 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.118582    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88dq9_d8131089-a28e-46d5-b800-de5c8e83457b/kube-multus-additional-cni-plugins/0.log"
Jul 18 02:34:51.138779 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.138753    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88dq9_d8131089-a28e-46d5-b800-de5c8e83457b/egress-router-binary-copy/0.log"
Jul 18 02:34:51.157964 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.157939    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88dq9_d8131089-a28e-46d5-b800-de5c8e83457b/cni-plugins/0.log"
Jul 18 02:34:51.176739 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.176710    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88dq9_d8131089-a28e-46d5-b800-de5c8e83457b/bond-cni-plugin/0.log"
Jul 18 02:34:51.196267 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.196228    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88dq9_d8131089-a28e-46d5-b800-de5c8e83457b/routeoverride-cni/0.log"
Jul 18 02:34:51.215783 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.215760    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88dq9_d8131089-a28e-46d5-b800-de5c8e83457b/whereabouts-cni-bincopy/0.log"
Jul 18 02:34:51.235907 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.235878    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88dq9_d8131089-a28e-46d5-b800-de5c8e83457b/whereabouts-cni/0.log"
Jul 18 02:34:51.572760 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.572714    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jm445_8222da46-c8c4-4d81-b240-a4455631215c/kube-multus/0.log"
Jul 18 02:34:51.715001 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.714970    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-npj5j_46d91c18-6dcf-4cb4-93a8-38d06290aa0a/network-metrics-daemon/0.log"
Jul 18 02:34:51.731369 ip-10-0-142-56 kubenswrapper[2580]: I0718 02:34:51.731343    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-npj5j_46d91c18-6dcf-4cb4-93a8-38d06290aa0a/kube-rbac-proxy/0.log"