Jul 21 04:07:28.719163 ip-10-0-140-142 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 21 04:07:28.719175 ip-10-0-140-142 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 21 04:07:28.719182 ip-10-0-140-142 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 21 04:07:28.719436 ip-10-0-140-142 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 21 04:07:38.841461 ip-10-0-140-142 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 21 04:07:38.841480 ip-10-0-140-142 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 90e3ced0ccc64d69908f776489731b2f --
Jul 21 04:09:38.160314 ip-10-0-140-142 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 04:09:38.552426 ip-10-0-140-142 kubenswrapper[2546]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 04:09:38.552426 ip-10-0-140-142 kubenswrapper[2546]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 04:09:38.552426 ip-10-0-140-142 kubenswrapper[2546]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 04:09:38.552426 ip-10-0-140-142 kubenswrapper[2546]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 04:09:38.552426 ip-10-0-140-142 kubenswrapper[2546]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 04:09:38.553189 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.552491    2546 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 04:09:38.558877 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558859    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 04:09:38.558877 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558877    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558880    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558884    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558887    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558891    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558894    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558896    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558899    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558902    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558905    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558908    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558910    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558913    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558916    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558919    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558921    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558928    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558931    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558933    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558936    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 04:09:38.558948 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558939    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558944    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558947    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558952    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558956    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558959    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558962    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558964    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558967    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558970    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558973    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558975    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558978    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558981    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558983    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558986    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558988    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558991    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558993    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 04:09:38.559462 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558996    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.558998    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559001    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559014    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559017    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559021    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559024    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559026    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559029    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559032    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559035    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559037    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559040    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559043    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559045    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559048    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559051    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559054    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559057    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559061    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 04:09:38.559911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559066    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559069    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559072    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559074    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559077    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559080    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559082    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559085    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559088    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559090    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559093    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559096    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559099    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559102    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559104    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559107    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559111    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559113    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559116    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559119    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 04:09:38.560520 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559121    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559124    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559126    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559130    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559133    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559135    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559138    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559142    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559144    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559153    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559157    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559160    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559162    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559166    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559169    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559171    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559174    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559177    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559179    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559888    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 04:09:38.561289 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559895    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559899    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559903    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559906    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559909    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559912    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559915    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559918    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559921    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559924    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559927    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559930    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559933    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559935    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559938    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559941    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559951    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559953    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559957    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559961    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 04:09:38.561843 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559964    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559967    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559969    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559972    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559974    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559977    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559979    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559982    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559985    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559987    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559990    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559992    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559995    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.559998    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560000    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560019    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560022    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560025    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560028    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560031    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 04:09:38.562434 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560033    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560036    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560039    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560041    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560044    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560048    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560052    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560056    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560060    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560063    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560066    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560071    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560074    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560077    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560080    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560083    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560085    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560088    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 04:09:38.562947 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560093    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560096    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560099    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560102    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560105    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560108    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560110    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560113    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560116    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560120    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560123    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560125    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560128    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560130    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560133    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560135    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560138    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560141    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560143    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560146    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 04:09:38.563415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560149    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560152    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560154    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560157    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560200    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560246    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560251    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560256    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560261    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.560265    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561083    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561090    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561095    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561099    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561104    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561108    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561113    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561117    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561128    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.561133    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 04:09:38.563894 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562205    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562218    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562222    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562224    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562228    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562231    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562234    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562237    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562239    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562242    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562245    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562248    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562250    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562253    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562256    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562258    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562261    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562264    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562266    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562270    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 04:09:38.564397 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562272    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562275    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562278    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562280    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562282    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562285    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562287    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562290    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562292    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562295    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562297    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562300    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562303    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562306    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562309    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562311    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562314    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562316    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562319    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562321    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 04:09:38.564915 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562324    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562326    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562329    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562331    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562334    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562337    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562339    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562342    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562344    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562347    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562349    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562352    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562355    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562358    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562360    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562363    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562365    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562368    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562370    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562373    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 04:09:38.565428 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562375    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562395    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562400    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562403    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562406    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562409    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562412    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562415    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562418    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562421    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562424    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562426    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562429    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562431    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562435    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562439    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562442    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562445    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562448    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562451    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 04:09:38.565911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562453    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562456    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562459    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562462    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562465    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562468    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562470    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562473    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562476    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562478    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562480    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562484    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562488    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562490    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562493    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562495    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562498    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562501    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.562504    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562570    2546 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 04:09:38.566447 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562577    2546 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562584    2546 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562589    2546 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562593    2546 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562597    2546 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562601    2546 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562605    2546 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562609    2546 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562611    2546 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562615    2546 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562618    2546 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562621    2546 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562624    2546 flags.go:64] FLAG: --cgroup-root=""
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562627    2546 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562630    2546 flags.go:64] FLAG: --client-ca-file=""
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562633    2546 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562635    2546 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562639    2546 flags.go:64] FLAG: --cluster-domain=""
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562642    2546 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562645    2546 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562648    2546 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562651    2546 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562656    2546 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 04:09:38.566912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562658    2546 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562661    2546 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562664    2546 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562667    2546 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562670    2546 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562673    2546 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562676    2546 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562679    2546 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562688    2546 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562692    2546 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562695    2546 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562698    2546 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562701    2546 flags.go:64] FLAG: --enable-server="true"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562703    2546 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562708    2546 flags.go:64] FLAG: --event-burst="100"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562711    2546 flags.go:64] FLAG: --event-qps="50"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562714    2546 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562717    2546 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562720    2546 flags.go:64] FLAG: --eviction-hard=""
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562723    2546 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562726    2546 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562729    2546 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562732    2546 flags.go:64] FLAG: --eviction-soft=""
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562735    2546 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562737    2546 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 04:09:38.567494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562740    2546 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562743    2546 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562746    2546 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562748    2546 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562751    2546 flags.go:64] FLAG: --feature-gates=""
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562755    2546 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562758    2546 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562761    2546 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562764    2546 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562767    2546 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562770    2546 flags.go:64] FLAG: --help="false"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562772    2546 flags.go:64] FLAG: --hostname-override="ip-10-0-140-142.ec2.internal"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562776    2546 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562780    2546 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562783    2546 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562786    2546 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562790    2546 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562794    2546 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562797    2546 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562800    2546 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562803    2546 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562805    2546 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562808    2546 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562811    2546 flags.go:64] FLAG: --kube-reserved=""
Jul 21 04:09:38.568164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562814    2546 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562817    2546 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562820    2546 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562822    2546 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562825    2546 flags.go:64] FLAG: --lock-file=""
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562828    2546 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562831    2546 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562834    2546 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562839    2546 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562842    2546 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562845    2546 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562848    2546 flags.go:64] FLAG: --logging-format="text"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562851    2546 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562854    2546 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562857    2546 flags.go:64] FLAG: --manifest-url=""
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562860    2546 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562864    2546 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562867    2546 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562871    2546 flags.go:64] FLAG: --max-pods="110"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562874    2546 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562879    2546 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562883    2546 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562886    2546 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562889    2546 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562892    2546 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 04:09:38.568734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562894    2546 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562902    2546 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562905    2546 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562908    2546 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562911    2546 flags.go:64] FLAG: --pod-cidr=""
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562914    2546 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562920    2546 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562923    2546 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562926    2546 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562929    2546 flags.go:64] FLAG: --port="10250"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562932    2546 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562935    2546 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e992a27b573fe586"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562938    2546 flags.go:64] FLAG: --qos-reserved=""
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562943    2546 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562946    2546 flags.go:64] FLAG: --register-node="true"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562949    2546 flags.go:64] FLAG: --register-with-taints=""
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562952    2546 flags.go:64] FLAG: --registry-burst="10"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562955    2546 flags.go:64] FLAG: --registry-qps="5"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562958    2546 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562961    2546 flags.go:64] FLAG: --reserved-memory=""
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562965    2546 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562968    2546 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562971    2546 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562974    2546 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562976    2546 flags.go:64] FLAG: --runonce="false"
Jul 21 04:09:38.569376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562979    2546 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562982    2546 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562985    2546 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562989    2546 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562992    2546 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562995    2546 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.562998    2546 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563001    2546 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563018    2546 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563022    2546 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563025    2546 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563027    2546 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563031    2546 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563034    2546 flags.go:64] FLAG: --system-cgroups=""
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563036    2546 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563042    2546 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563045    2546 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563048    2546 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563052    2546 flags.go:64] FLAG: --tls-min-version=""
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563055    2546 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563058    2546 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563061    2546 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563064    2546 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563067    2546 flags.go:64] FLAG: --v="2"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563071    2546 flags.go:64] FLAG: --version="false"
Jul 21 04:09:38.569999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563075    2546 flags.go:64] FLAG: --vmodule=""
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563079    2546 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.563082    2546 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563214    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563218    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563222    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563225    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563227    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563230    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563233    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563237    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563240    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563242    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563245    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563247    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563250    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563253    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563256    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563258    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563261    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 04:09:38.570613 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563264    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563266    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563269    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563271    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563274    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563276    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563279    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563282    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563284    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563287    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563290    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563292    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563295    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563297    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563300    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563303    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563305    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563308    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563310    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563313    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 04:09:38.571118 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563315    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563318    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563322    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563324    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563327    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563329    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563332    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563335    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563337    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563340    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563343    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563346    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563348    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563351    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563353    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563356    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563358    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563361    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563365    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 04:09:38.571649 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563369    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563371    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563374    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563377    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563380    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563383    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563386    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563389    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563392    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563395    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563397    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563400    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563403    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563406    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563409    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563412    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563415    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563417    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563420    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 04:09:38.572126 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563423    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563425    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563428    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563431    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563433    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563438    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563442    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563444    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563447    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563449    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563452    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563454    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563458    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563462    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563465    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563467    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563469    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563472    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563474    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 04:09:38.572581 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563477    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563480    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563482    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563484    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.563487    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.564089    2546 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.570804    2546 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.570821    2546 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.571678    2546 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.571686    2546 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571745    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571750    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571754    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571757    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571760    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 04:09:38.573040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571763    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571765    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571768    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571770    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571773    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571777    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571781    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571784    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571787    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571790    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571794    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571796    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571800    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571803    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571805    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571808    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571810    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571813    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571816    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 04:09:38.573415 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571818    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571822    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571826    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571829    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571832    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571835    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571838    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571841    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571844    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571846    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571849    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571852    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571855    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571857    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571860    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571862    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571867    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571870    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571873    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 04:09:38.573861 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571876    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571878    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571881    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571884    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571887    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571889    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571892    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571895    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571897    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571900    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571902    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571905    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571907    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571910    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571913    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571915    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571917    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571920    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571922    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 04:09:38.574322 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571926    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571929    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571931    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571934    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571937    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571939    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571942    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571944    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571947    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571949    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571952    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571955    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571958    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571960    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571962    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571965    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571967    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571970    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571972    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571975    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 04:09:38.574835 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571977    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571980    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571982    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571985    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571987    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571990    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571992    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571995    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.571997    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572000    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572002    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572019    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572022    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572025    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572028    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572031    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 04:09:38.575460 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572034    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.572039    2546 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572154    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572159    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572162    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572165    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572168    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572171    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572174    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572177    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572180    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572183    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572185    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572188    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572190    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 04:09:38.575864 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572193    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572195    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572198    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572200    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572203    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572205    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572208    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572210    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572213    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572216    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572219    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572221    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572224    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572226    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572229    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572232    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572235    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572239    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572242    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572245    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 04:09:38.576350 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572248    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572252    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572255    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572258    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572261    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572263    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572266    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572268    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572271    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572274    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572277    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572280    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572285    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572287    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572290    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572292    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572295    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572298    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572300    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 04:09:38.576818 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572303    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572306    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572309    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572312    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572314    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572317    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572319    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572322    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572325    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572328    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572331    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572333    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572336    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572338    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572341    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572343    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572346    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572348    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572351    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572353    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 04:09:38.577317 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572356    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572358    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572361    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572363    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572366    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572373    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572376    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572378    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572381    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572383    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572386    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572388    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572391    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572395    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572397    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572400    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572403    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572405    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572408    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 04:09:38.577893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572410    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572413    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572416    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572419    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572422    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572424    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572427    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:38.572429    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.572434    2546 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.572553    2546 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.574516    2546 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.575816    2546 server.go:1019] "Starting client certificate rotation"
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.575935    2546 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 04:09:38.578366 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.575979    2546 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 04:09:38.578853 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.578174    2546 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 04:09:38.579314 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.579295    2546 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 04:09:38.596391 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.596372    2546 log.go:25] "Validated CRI v1 runtime API"
Jul 21 04:09:38.598916 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.598897    2546 log.go:25] "Validated CRI v1 image API"
Jul 21 04:09:38.599764 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.599743    2546 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 04:09:38.602625 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.602598    2546 fs.go:135] Filesystem UUIDs: map[57b530cf-7fd4-4ebe-861c-e853f179aa0e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c1e411db-a9e1-4d4f-b15b-cd946848f97f:/dev/nvme0n1p3]
Jul 21 04:09:38.602686 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.602625    2546 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 04:09:38.606248 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.606224    2546 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 04:09:38.609317 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.609215    2546 manager.go:216] Machine: {Timestamp:2026-07-21 04:09:38.608076537 +0000 UTC m=+0.333672862 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3078146 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec270433a2bac0ccc3f439aa0337585a SystemUUID:ec270433-a2ba-c0cc-c3f4-39aa0337585a BootID:90e3ced0-ccc6-4d69-908f-776489731b2f Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ba:f8:68:72:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ba:f8:68:72:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:aa:92:8f:a2:37 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 04:09:38.609317 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.609312    2546 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 04:09:38.609430 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.609393    2546 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 04:09:38.610367 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.610345    2546 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 04:09:38.610496 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.610371    2546 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-142.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 04:09:38.611565 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.611555    2546 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 04:09:38.611598 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.611569    2546 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 04:09:38.611598 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.611577    2546 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 04:09:38.612346 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.612336    2546 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 04:09:38.613613 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.613602    2546 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 04:09:38.616707 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.616696    2546 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 04:09:38.616815 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.616805    2546 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 04:09:38.616849 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.616834    2546 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 04:09:38.616849 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.616843    2546 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 04:09:38.616902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.616853    2546 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 04:09:38.616902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.616861    2546 kubelet.go:398] "Adding apiserver pod source"
Jul 21 04:09:38.616902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.616869    2546 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 04:09:38.617282 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.617261    2546 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 04:09:38.617282 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.617279    2546 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 04:09:38.619453 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.619437    2546 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 04:09:38.621358 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.621343    2546 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 04:09:38.621419 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.621376    2546 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 04:09:38.622377 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622365    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 04:09:38.622431 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622385    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 04:09:38.622431 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622392    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 04:09:38.622431 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622397    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 04:09:38.622431 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622405    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 04:09:38.622431 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622411    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 04:09:38.622698 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622690    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 04:09:38.622732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622699    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 04:09:38.622732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622706    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 04:09:38.622732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622711    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 04:09:38.622732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622720    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 04:09:38.622732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.622727    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 04:09:38.623436 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.623426    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 04:09:38.623467 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.623438    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 04:09:38.625685 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.625653    2546 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-142.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 04:09:38.625775 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.625754    2546 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 04:09:38.625775 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.625745    2546 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 04:09:38.626906 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.626893    2546 server.go:1268] "Started kubelet"
Jul 21 04:09:38.626997 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.626949    2546 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 04:09:38.627261 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.627096    2546 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 04:09:38.627433 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.627421    2546 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 04:09:38.627976 ip-10-0-140-142 systemd[1]: Started Kubernetes Kubelet.
Jul 21 04:09:38.628979 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.628928    2546 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 04:09:38.629142 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.629128    2546 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 04:09:38.629767 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.629750    2546 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2j64c"
Jul 21 04:09:38.632440 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.631282    2546 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-142.ec2.internal.18c4329780bcc116  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-142.ec2.internal,UID:ip-10-0-140-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-142.ec2.internal,},FirstTimestamp:2026-07-21 04:09:38.626863382 +0000 UTC m=+0.352459707,LastTimestamp:2026-07-21 04:09:38.626863382 +0000 UTC m=+0.352459707,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-142.ec2.internal,}"
Jul 21 04:09:38.634167 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.634150    2546 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 04:09:38.635427 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.635409    2546 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 04:09:38.635513 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.635423    2546 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 04:09:38.635644 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.635626    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:38.635734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.635714    2546 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2j64c"
Jul 21 04:09:38.636197 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636183    2546 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 04:09:38.636351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636335    2546 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 04:09:38.636351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636185    2546 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 04:09:38.636482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636339    2546 factory.go:55] Registering systemd factory
Jul 21 04:09:38.636482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636370    2546 factory.go:223] Registration of the systemd container factory successfully
Jul 21 04:09:38.636482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636443    2546 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 04:09:38.636482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636454    2546 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 04:09:38.636790 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636775    2546 factory.go:153] Registering CRI-O factory
Jul 21 04:09:38.636852 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636794    2546 factory.go:223] Registration of the crio container factory successfully
Jul 21 04:09:38.636904 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636858    2546 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 04:09:38.636904 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636882    2546 factory.go:103] Registering Raw factory
Jul 21 04:09:38.636904 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.636896    2546 manager.go:1196] Started watching for new ooms in manager
Jul 21 04:09:38.637726 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.637712    2546 manager.go:349] Starting recovery of all containers
Jul 21 04:09:38.638646 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.638447    2546 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 04:09:38.639204 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.638562    2546 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 04:09:38.651633 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.651501    2546 manager.go:354] Recovery completed
Jul 21 04:09:38.656019 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.655995    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 04:09:38.658079 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.658064    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 04:09:38.658125 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.658089    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 04:09:38.658125 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.658100    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Jul 21 04:09:38.658683 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.658665    2546 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 04:09:38.658683 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.658682    2546 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 04:09:38.658811 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.658699    2546 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 04:09:38.658811 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.658709    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 04:09:38.659454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.659434    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 04:09:38.659524 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.659470    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 04:09:38.659524 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.659486    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Jul 21 04:09:38.659967 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.659954    2546 policy_none.go:49] "None policy: Start"
Jul 21 04:09:38.660053 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.659973    2546 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 04:09:38.660053 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.659988    2546 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 04:09:38.661212 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.661199    2546 policy_none.go:47] "Start"
Jul 21 04:09:38.692566 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.692549    2546 manager.go:337] "Starting Device Plugin manager"
Jul 21 04:09:38.692656 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.692587    2546 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 04:09:38.692656 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.692599    2546 server.go:87] "Starting device plugin registration server"
Jul 21 04:09:38.692874 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.692861    2546 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 04:09:38.692922 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.692875    2546 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 04:09:38.693120 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.693067    2546 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 04:09:38.693189 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.693144    2546 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 04:09:38.693189 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.693153    2546 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 04:09:38.693804 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.693781    2546 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 04:09:38.693895 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.693842    2546 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:38.793499 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.793467    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 04:09:38.794584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.794433    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 04:09:38.794584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.794468    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 04:09:38.794584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.794488    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Jul 21 04:09:38.794584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.794518    2546 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-142.ec2.internal"
Jul 21 04:09:38.803172 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.803119    2546 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-142.ec2.internal"
Jul 21 04:09:38.803172 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.803142    2546 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-142.ec2.internal\": node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:38.814833 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.814804    2546 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 04:09:38.816189 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.816175    2546 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 04:09:38.816257 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.816210    2546 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 04:09:38.816257 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.816229    2546 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 04:09:38.816360 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.816264    2546 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 04:09:38.817828 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.817807    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:38.817828 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.817820    2546 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 04:09:38.917101 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.917060    2546 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"]
Jul 21 04:09:38.917291 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.917159    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 04:09:38.917895 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.917880    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:38.918233 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.918219    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 04:09:38.918299 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.918250    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 04:09:38.918299 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.918260    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Jul 21 04:09:38.919933 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.919918    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 04:09:38.920106 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.920065    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:38.920106 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.920099    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 04:09:38.920616 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.920598    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 04:09:38.920693 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.920627    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 04:09:38.920693 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.920607    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 04:09:38.920693 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.920657    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 04:09:38.920693 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.920676    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Jul 21 04:09:38.920693 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.920639    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Jul 21 04:09:38.922041 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.922000    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:38.922158 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.922054    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 04:09:38.922658 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.922645    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 04:09:38.922720 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.922664    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 04:09:38.922720 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.922674    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Jul 21 04:09:38.936583 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.936558    2546 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-142.ec2.internal\" not found" node="ip-10-0-140-142.ec2.internal"
Jul 21 04:09:38.938415 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.938393    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3908927fc6f60eab4bf00fa616e3360a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3908927fc6f60eab4bf00fa616e3360a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:38.938505 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.938427    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3908927fc6f60eab4bf00fa616e3360a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3908927fc6f60eab4bf00fa616e3360a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:38.938505 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:38.938480    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f5f2842cd541864d0773b17e6d2e6e62-config\") pod \"kube-apiserver-proxy-ip-10-0-140-142.ec2.internal\" (UID: \"f5f2842cd541864d0773b17e6d2e6e62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:38.940915 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:38.940895    2546 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-142.ec2.internal\" not found" node="ip-10-0-140-142.ec2.internal"
Jul 21 04:09:39.018578 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:39.018549    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:39.038929 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.038890    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3908927fc6f60eab4bf00fa616e3360a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3908927fc6f60eab4bf00fa616e3360a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:39.038929 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.038928    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3908927fc6f60eab4bf00fa616e3360a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3908927fc6f60eab4bf00fa616e3360a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:39.039119 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.038946    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f5f2842cd541864d0773b17e6d2e6e62-config\") pod \"kube-apiserver-proxy-ip-10-0-140-142.ec2.internal\" (UID: \"f5f2842cd541864d0773b17e6d2e6e62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:39.039119 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.038979    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3908927fc6f60eab4bf00fa616e3360a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3908927fc6f60eab4bf00fa616e3360a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:39.039119 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.038988    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3908927fc6f60eab4bf00fa616e3360a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3908927fc6f60eab4bf00fa616e3360a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:39.039119 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.039037    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f5f2842cd541864d0773b17e6d2e6e62-config\") pod \"kube-apiserver-proxy-ip-10-0-140-142.ec2.internal\" (UID: \"f5f2842cd541864d0773b17e6d2e6e62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:39.119344 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:39.119267    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:39.219788 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:39.219750    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:39.240155 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.240135    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:39.244462 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.244287    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:39.320117 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:39.320083    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:39.420670 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:39.420607    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:39.521145 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:39.521116    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:39.575762 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.575739    2546 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 04:09:39.576183 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.575873    2546 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 04:09:39.622215 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:39.622190    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:39.641568 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.641538    2546 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 04:04:39 +0000 UTC" deadline="2028-04-25 02:24:29.17982109 +0000 UTC"
Jul 21 04:09:39.641568 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:39.641563    2546 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15454h14m49.538260398s"
Jul 21 04:09:39.723076 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:39.722991    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:41.163151 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:41.163102    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5f2842cd541864d0773b17e6d2e6e62.slice/crio-8b10e039bebf25a5f51f94214bab428efaacb172e901a676ee3bf8695e621053 WatchSource:0}: Error finding container 8b10e039bebf25a5f51f94214bab428efaacb172e901a676ee3bf8695e621053: Status 404 returned error can't find the container with id 8b10e039bebf25a5f51f94214bab428efaacb172e901a676ee3bf8695e621053
Jul 21 04:09:41.163276 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:41.163164    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Jul 21 04:09:41.163394 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:09:41.163380    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3908927fc6f60eab4bf00fa616e3360a.slice/crio-d2b03069246e8ea147a5b14a7e074d8cff012f799fcffb6a307c5e91c5b001b1 WatchSource:0}: Error finding container d2b03069246e8ea147a5b14a7e074d8cff012f799fcffb6a307c5e91c5b001b1: Status 404 returned error can't find the container with id d2b03069246e8ea147a5b14a7e074d8cff012f799fcffb6a307c5e91c5b001b1
Jul 21 04:09:41.167162 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.167144    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 04:09:41.181305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.181287    2546 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 04:09:41.276361 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.276332    2546 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:41.286818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.286793    2546 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 04:09:41.288638 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.288622    2546 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Jul 21 04:09:41.296302 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.296282    2546 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 04:09:41.347141 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.347060    2546 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 04:09:41.500187 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.500159    2546 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 04:09:41.957412 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.957383    2546 apiserver.go:52] "Watching apiserver"
Jul 21 04:09:41.966442 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.966417    2546 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 04:09:41.966983 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.966943    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-glqxn","kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph","openshift-cluster-node-tuning-operator/tuned-r582g","openshift-dns/node-resolver-82nbl","openshift-image-registry/node-ca-vrmch","openshift-multus/multus-gjjzt","openshift-network-diagnostics/network-check-target-xv2m6","kube-system/global-pull-secret-syncer-75ljs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal","openshift-multus/multus-additional-cni-plugins-x5qqx","openshift-multus/network-metrics-daemon-c6kf7","openshift-network-operator/iptables-alerter-f8d25","openshift-ovn-kubernetes/ovnkube-node-xp2zl"]
Jul 21 04:09:41.968919 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.968899    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.970661 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.970594    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:41.971863 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.971472    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 04:09:41.971863 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.971530    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 04:09:41.971863 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.971472    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 04:09:41.971863 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.971738    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4fpgg\""
Jul 21 04:09:41.972102 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.971984    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:41.972498 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.972478    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 04:09:41.974578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.973166    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 04:09:41.974578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.973474    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 04:09:41.974578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.973594    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.974578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.973704    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r59jz\""
Jul 21 04:09:41.974578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.974219    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 04:09:41.974578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.974282    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 04:09:41.974578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.974380    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 04:09:41.974578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.974449    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 04:09:41.974578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.974514    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w5dpb\""
Jul 21 04:09:41.975026 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.974759    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:41.977307 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.975838    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 04:09:41.977307 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.975957    2546 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 04:09:41.977307 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.975836    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7gfs7\""
Jul 21 04:09:41.977878 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.977858    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 04:09:41.978315 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.978243    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 04:09:41.978408 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.978367    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 04:09:41.978465 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.978433    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wb4gj\""
Jul 21 04:09:41.979595 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.978883    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:41.983006 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.981259    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 04:09:41.983006 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.981401    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2j8hp\""
Jul 21 04:09:41.983006 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.981423    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:41.983006 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:41.981644    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:09:41.983006 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.981786    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 04:09:41.983006 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.982003    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 04:09:41.986132 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.986097    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.986959    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-etc-kubernetes\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987013    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9k8l\" (UniqueName: \"kubernetes.io/projected/b50b377c-8166-4fe6-9c61-e739659c12c7-kube-api-access-d9k8l\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987046    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-tuned\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987077    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b4rm\" (UniqueName: \"kubernetes.io/projected/78a71799-4b84-44a2-a968-2b591c4f785d-kube-api-access-7b4rm\") pod \"iptables-alerter-f8d25\" (UID: \"78a71799-4b84-44a2-a968-2b591c4f785d\") " pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987148    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-systemd\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987173    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-lib-modules\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987208    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d72963f6-4559-479b-b8d1-633a1cab9d0d-tmp\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987237    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-cni-dir\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987275    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-registration-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987307    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-modprobe-d\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987336    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-sysconfig\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987359    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-run\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987396    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-sys\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987436    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-os-release\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987467    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-run-k8s-cni-cncf-io\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987502    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-daemon-config\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987533    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtp4z\" (UniqueName: \"kubernetes.io/projected/3798e436-1a3e-47a4-8911-724c05dc7e53-kube-api-access-mtp4z\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987566    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/78a71799-4b84-44a2-a968-2b591c4f785d-iptables-alerter-script\") pod \"iptables-alerter-f8d25\" (UID: \"78a71799-4b84-44a2-a968-2b591c4f785d\") " pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987630    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-socket-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987794    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987835    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-cnibin\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987865    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-socket-dir-parent\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987897    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-var-lib-cni-bin\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987926    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987955    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-sys-fs\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.987986    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-kubernetes\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988029    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-sysctl-conf\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988085    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-var-lib-kubelet\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988139    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3798e436-1a3e-47a4-8911-724c05dc7e53-cni-binary-copy\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988181    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-var-lib-cni-multus\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988205    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-var-lib-kubelet\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988234    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-conf-dir\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.988861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988280    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-device-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988308    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-host\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988342    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmntv\" (UniqueName: \"kubernetes.io/projected/2140b0ce-d757-4b6d-b07a-82e3b8e15f90-kube-api-access-pmntv\") pod \"node-resolver-82nbl\" (UID: \"2140b0ce-d757-4b6d-b07a-82e3b8e15f90\") " pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988411    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-system-cni-dir\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988441    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-run-netns\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988474    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-run-multus-certs\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988502    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-sysctl-d\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988519    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q782l\" (UniqueName: \"kubernetes.io/projected/d72963f6-4559-479b-b8d1-633a1cab9d0d-kube-api-access-q782l\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988534    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2140b0ce-d757-4b6d-b07a-82e3b8e15f90-tmp-dir\") pod \"node-resolver-82nbl\" (UID: \"2140b0ce-d757-4b6d-b07a-82e3b8e15f90\") " pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988552    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b761d554-5f1b-4da7-b197-d4cd24446e11-host\") pod \"node-ca-vrmch\" (UID: \"b761d554-5f1b-4da7-b197-d4cd24446e11\") " pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988567    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-hostroot\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988600    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78a71799-4b84-44a2-a968-2b591c4f785d-host-slash\") pod \"iptables-alerter-f8d25\" (UID: \"78a71799-4b84-44a2-a968-2b591c4f785d\") " pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988622    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2140b0ce-d757-4b6d-b07a-82e3b8e15f90-hosts-file\") pod \"node-resolver-82nbl\" (UID: \"2140b0ce-d757-4b6d-b07a-82e3b8e15f90\") " pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988700    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b761d554-5f1b-4da7-b197-d4cd24446e11-serviceca\") pod \"node-ca-vrmch\" (UID: \"b761d554-5f1b-4da7-b197-d4cd24446e11\") " pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:41.989642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.988729    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rqp7\" (UniqueName: \"kubernetes.io/projected/b761d554-5f1b-4da7-b197-d4cd24446e11-kube-api-access-8rqp7\") pod \"node-ca-vrmch\" (UID: \"b761d554-5f1b-4da7-b197-d4cd24446e11\") " pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:41.990289 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.989809    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 04:09:41.991209 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.991193    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lmg9s\""
Jul 21 04:09:41.991354 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.991337    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:41.992806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.992787    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:41.992895 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:41.992847    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:41.992895 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.992867    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 04:09:41.993691 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.993673    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 04:09:41.994439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.993824    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vk4md\""
Jul 21 04:09:41.994439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.993911    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 04:09:41.994439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.993998    2546 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 04:09:41.994439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.994145    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 04:09:41.994439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.994217    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 04:09:41.994439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.994328    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 04:09:41.995601 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.995095    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 04:09:41.996624 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.995895    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:41.997616 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.997595    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:41.997716 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:41.997666    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:09:41.998095 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.998081    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nnsbl\""
Jul 21 04:09:41.998258 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.998241    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 04:09:41.998933 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:41.998746    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 04:09:42.009077 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.009059    2546 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bzjpw"
Jul 21 04:09:42.019897 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.019877    2546 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bzjpw"
Jul 21 04:09:42.077972 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.077914    2546 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 04:09:42.081330 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.081305    2546 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 04:09:42.089071 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.088979    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-run-openvswitch\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.089071 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089014    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7e1e854-9d5b-4763-89f1-9829d53da287-ovnkube-script-lib\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.089071 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089046    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-registration-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.089300 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089152    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-registration-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.089300 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089207    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-var-lib-openvswitch\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.089300 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089239    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-run-ovn\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.089300 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089262    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7e1e854-9d5b-4763-89f1-9829d53da287-ovnkube-config\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.089495 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089309    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-daemon-config\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.089495 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089332    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtp4z\" (UniqueName: \"kubernetes.io/projected/3798e436-1a3e-47a4-8911-724c05dc7e53-kube-api-access-mtp4z\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.089495 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089355    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-socket-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.089622 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089574    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-sysctl-conf\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.089622 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089598    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b761d554-5f1b-4da7-b197-d4cd24446e11-host\") pod \"node-ca-vrmch\" (UID: \"b761d554-5f1b-4da7-b197-d4cd24446e11\") " pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:42.089674 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089636    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0037790b-0ec5-4bd1-ac34-35d830df61d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.089674 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089656    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-cnibin\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.089674 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089670    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-var-lib-cni-bin\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.089806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089730    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-cnibin\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.089806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089750    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.089806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089766    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-kubernetes\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.089806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089780    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-var-lib-kubelet\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.089806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089793    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmntv\" (UniqueName: \"kubernetes.io/projected/2140b0ce-d757-4b6d-b07a-82e3b8e15f90-kube-api-access-pmntv\") pod \"node-resolver-82nbl\" (UID: \"2140b0ce-d757-4b6d-b07a-82e3b8e15f90\") " pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:42.089943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089811    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-var-lib-cni-multus\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.089943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089827    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-device-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.089943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089843    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/30699fbf-ab92-43f7-8d71-d4f04fde4381-konnectivity-ca\") pod \"konnectivity-agent-glqxn\" (UID: \"30699fbf-ab92-43f7-8d71-d4f04fde4381\") " pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:09:42.089943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089857    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.089943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089874    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-system-cni-dir\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.089943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089926    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2140b0ce-d757-4b6d-b07a-82e3b8e15f90-tmp-dir\") pod \"node-resolver-82nbl\" (UID: \"2140b0ce-d757-4b6d-b07a-82e3b8e15f90\") " pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:42.089943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089940    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rqp7\" (UniqueName: \"kubernetes.io/projected/b761d554-5f1b-4da7-b197-d4cd24446e11-kube-api-access-8rqp7\") pod \"node-ca-vrmch\" (UID: \"b761d554-5f1b-4da7-b197-d4cd24446e11\") " pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089959    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-node-log\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089973    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-log-socket\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.089988    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-hostroot\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090010    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78a71799-4b84-44a2-a968-2b591c4f785d-host-slash\") pod \"iptables-alerter-f8d25\" (UID: \"78a71799-4b84-44a2-a968-2b591c4f785d\") " pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090024    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2140b0ce-d757-4b6d-b07a-82e3b8e15f90-hosts-file\") pod \"node-resolver-82nbl\" (UID: \"2140b0ce-d757-4b6d-b07a-82e3b8e15f90\") " pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090037    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9k8l\" (UniqueName: \"kubernetes.io/projected/b50b377c-8166-4fe6-9c61-e739659c12c7-kube-api-access-d9k8l\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090052    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-tuned\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090078    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0037790b-0ec5-4bd1-ac34-35d830df61d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090127    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-systemd-units\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.090148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090143    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-slash\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090157    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-922ck\" (UniqueName: \"kubernetes.io/projected/c7e1e854-9d5b-4763-89f1-9829d53da287-kube-api-access-922ck\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090175    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-lib-modules\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090190    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090204    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-os-release\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090230    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090323    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw5v8\" (UniqueName: \"kubernetes.io/projected/15bb69f1-6e84-48be-92fb-fc48362465b7-kube-api-access-gw5v8\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090340    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-cni-dir\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090355    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-modprobe-d\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090371    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-sysconfig\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090385    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-run\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090399    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-sys\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.090428 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090421    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/30699fbf-ab92-43f7-8d71-d4f04fde4381-agent-certs\") pod \"konnectivity-agent-glqxn\" (UID: \"30699fbf-ab92-43f7-8d71-d4f04fde4381\") " pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090438    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-os-release\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090453    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-run-k8s-cni-cncf-io\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090470    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/78a71799-4b84-44a2-a968-2b591c4f785d-iptables-alerter-script\") pod \"iptables-alerter-f8d25\" (UID: \"78a71799-4b84-44a2-a968-2b591c4f785d\") " pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090484    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090499    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q782l\" (UniqueName: \"kubernetes.io/projected/d72963f6-4559-479b-b8d1-633a1cab9d0d-kube-api-access-q782l\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090515    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0037790b-0ec5-4bd1-ac34-35d830df61d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090531    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090549    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-socket-dir-parent\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090564    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-sys-fs\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090578    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f4d15b6-a854-4feb-82a8-3fc155affffb-dbus\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090599    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-run-systemd\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090614    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-cni-netd\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090629    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7e1e854-9d5b-4763-89f1-9829d53da287-env-overrides\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090646    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3798e436-1a3e-47a4-8911-724c05dc7e53-cni-binary-copy\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090661    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-var-lib-kubelet\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090676    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-conf-dir\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.090793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090691    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-host\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090706    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctpxg\" (UniqueName: \"kubernetes.io/projected/0037790b-0ec5-4bd1-ac34-35d830df61d3-kube-api-access-ctpxg\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090721    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-kubelet\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090737    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090755    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-run-netns\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090770    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-run-multus-certs\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.090786    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-sysctl-d\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091226    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f4d15b6-a854-4feb-82a8-3fc155affffb-kubelet-config\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091240    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-run-netns\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091263    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-run-netns\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091228    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-run-multus-certs\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091291    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-system-cni-dir\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091312    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-sysconfig\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091330    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-var-lib-cni-multus\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091335    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-sysctl-d\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.091349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091344    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7e1e854-9d5b-4763-89f1-9829d53da287-ovn-node-metrics-cert\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091375    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-var-lib-cni-bin\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091375    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b761d554-5f1b-4da7-b197-d4cd24446e11-serviceca\") pod \"node-ca-vrmch\" (UID: \"b761d554-5f1b-4da7-b197-d4cd24446e11\") " pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091393    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091420    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-etc-openvswitch\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091432    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-var-lib-kubelet\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091435    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-socket-dir-parent\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091449    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-etc-kubernetes\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091477    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-kubernetes\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091478    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-sys-fs\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091478    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091494    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-lib-modules\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091514    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-cnibin\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091554    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-hostroot\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091571    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-sysctl-conf\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091588    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-cni-bin\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091608    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-os-release\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.092009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091609    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-cni-dir\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091625    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b4rm\" (UniqueName: \"kubernetes.io/projected/78a71799-4b84-44a2-a968-2b591c4f785d-kube-api-access-7b4rm\") pod \"iptables-alerter-f8d25\" (UID: \"78a71799-4b84-44a2-a968-2b591c4f785d\") " pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091632    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-run-k8s-cni-cncf-io\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091689    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-systemd\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091717    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d72963f6-4559-479b-b8d1-633a1cab9d0d-tmp\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091749    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-system-cni-dir\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091767    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2140b0ce-d757-4b6d-b07a-82e3b8e15f90-tmp-dir\") pod \"node-resolver-82nbl\" (UID: \"2140b0ce-d757-4b6d-b07a-82e3b8e15f90\") " pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091832    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78a71799-4b84-44a2-a968-2b591c4f785d-host-slash\") pod \"iptables-alerter-f8d25\" (UID: \"78a71799-4b84-44a2-a968-2b591c4f785d\") " pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091846    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-host\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091880    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2140b0ce-d757-4b6d-b07a-82e3b8e15f90-hosts-file\") pod \"node-resolver-82nbl\" (UID: \"2140b0ce-d757-4b6d-b07a-82e3b8e15f90\") " pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091336    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-device-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091423    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-sys\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092014    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-socket-dir\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092023    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b761d554-5f1b-4da7-b197-d4cd24446e11-serviceca\") pod \"node-ca-vrmch\" (UID: \"b761d554-5f1b-4da7-b197-d4cd24446e11\") " pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.091690    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-modprobe-d\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092068    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b761d554-5f1b-4da7-b197-d4cd24446e11-host\") pod \"node-ca-vrmch\" (UID: \"b761d554-5f1b-4da7-b197-d4cd24446e11\") " pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092069    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-run\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092141    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-conf-dir\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.092792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092148    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-host-var-lib-kubelet\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.093482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092157    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b50b377c-8166-4fe6-9c61-e739659c12c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.093482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092196    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3798e436-1a3e-47a4-8911-724c05dc7e53-etc-kubernetes\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.093482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092197    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-systemd\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.093482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092247    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3798e436-1a3e-47a4-8911-724c05dc7e53-multus-daemon-config\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.093482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092328    2546 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 04:09:42.093482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092468    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/78a71799-4b84-44a2-a968-2b591c4f785d-iptables-alerter-script\") pod \"iptables-alerter-f8d25\" (UID: \"78a71799-4b84-44a2-a968-2b591c4f785d\") " pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:42.093482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.092943    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3798e436-1a3e-47a4-8911-724c05dc7e53-cni-binary-copy\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.096187 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.096165    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d72963f6-4559-479b-b8d1-633a1cab9d0d-etc-tuned\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.096415 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.096395    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d72963f6-4559-479b-b8d1-633a1cab9d0d-tmp\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.109559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.109529    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9k8l\" (UniqueName: \"kubernetes.io/projected/b50b377c-8166-4fe6-9c61-e739659c12c7-kube-api-access-d9k8l\") pod \"aws-ebs-csi-driver-node-bh6ph\" (UID: \"b50b377c-8166-4fe6-9c61-e739659c12c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.110174 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.110016    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q782l\" (UniqueName: \"kubernetes.io/projected/d72963f6-4559-479b-b8d1-633a1cab9d0d-kube-api-access-q782l\") pod \"tuned-r582g\" (UID: \"d72963f6-4559-479b-b8d1-633a1cab9d0d\") " pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.111738 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.110430    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtp4z\" (UniqueName: \"kubernetes.io/projected/3798e436-1a3e-47a4-8911-724c05dc7e53-kube-api-access-mtp4z\") pod \"multus-gjjzt\" (UID: \"3798e436-1a3e-47a4-8911-724c05dc7e53\") " pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.111738 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.110530    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rqp7\" (UniqueName: \"kubernetes.io/projected/b761d554-5f1b-4da7-b197-d4cd24446e11-kube-api-access-8rqp7\") pod \"node-ca-vrmch\" (UID: \"b761d554-5f1b-4da7-b197-d4cd24446e11\") " pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:42.111738 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.110780    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b4rm\" (UniqueName: \"kubernetes.io/projected/78a71799-4b84-44a2-a968-2b591c4f785d-kube-api-access-7b4rm\") pod \"iptables-alerter-f8d25\" (UID: \"78a71799-4b84-44a2-a968-2b591c4f785d\") " pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:42.111940 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.111922    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmntv\" (UniqueName: \"kubernetes.io/projected/2140b0ce-d757-4b6d-b07a-82e3b8e15f90-kube-api-access-pmntv\") pod \"node-resolver-82nbl\" (UID: \"2140b0ce-d757-4b6d-b07a-82e3b8e15f90\") " pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:42.161457 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.161406    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal" event={"ID":"f5f2842cd541864d0773b17e6d2e6e62","Type":"ContainerStarted","Data":"8b10e039bebf25a5f51f94214bab428efaacb172e901a676ee3bf8695e621053"}
Jul 21 04:09:42.162737 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.162706    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal" event={"ID":"3908927fc6f60eab4bf00fa616e3360a","Type":"ContainerStarted","Data":"d2b03069246e8ea147a5b14a7e074d8cff012f799fcffb6a307c5e91c5b001b1"}
Jul 21 04:09:42.193149 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.192974    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7e1e854-9d5b-4763-89f1-9829d53da287-env-overrides\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193149 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193017    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctpxg\" (UniqueName: \"kubernetes.io/projected/0037790b-0ec5-4bd1-ac34-35d830df61d3-kube-api-access-ctpxg\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.193149 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193046    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-kubelet\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193149 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193068    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193149 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193097    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f4d15b6-a854-4feb-82a8-3fc155affffb-kubelet-config\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:42.193149 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193139    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-run-netns\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193162    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7e1e854-9d5b-4763-89f1-9829d53da287-ovn-node-metrics-cert\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193187    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-etc-openvswitch\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193210    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193234    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-cnibin\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193236    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-kubelet\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193255    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-cni-bin\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193266    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-run-netns\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193306    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-cni-bin\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193322    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193352    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-etc-openvswitch\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193375    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f4d15b6-a854-4feb-82a8-3fc155affffb-kubelet-config\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193416    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-cnibin\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193445    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-system-cni-dir\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193472    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-run-openvswitch\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193496    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7e1e854-9d5b-4763-89f1-9829d53da287-ovnkube-script-lib\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193507    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-system-cni-dir\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.193514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193520    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-var-lib-openvswitch\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193545    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-run-ovn\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193552    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-run-openvswitch\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193568    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7e1e854-9d5b-4763-89f1-9829d53da287-ovnkube-config\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193593    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-var-lib-openvswitch\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193597    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0037790b-0ec5-4bd1-ac34-35d830df61d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193630    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/30699fbf-ab92-43f7-8d71-d4f04fde4381-konnectivity-ca\") pod \"konnectivity-agent-glqxn\" (UID: \"30699fbf-ab92-43f7-8d71-d4f04fde4381\") " pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193631    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7e1e854-9d5b-4763-89f1-9829d53da287-env-overrides\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193653    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193659    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-run-ovn\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193678    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-node-log\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193702    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-log-socket\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193729    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0037790b-0ec5-4bd1-ac34-35d830df61d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193754    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-systemd-units\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193777    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-slash\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193800    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-922ck\" (UniqueName: \"kubernetes.io/projected/c7e1e854-9d5b-4763-89f1-9829d53da287-kube-api-access-922ck\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193826    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:42.194312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193849    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-os-release\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193874    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193898    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw5v8\" (UniqueName: \"kubernetes.io/projected/15bb69f1-6e84-48be-92fb-fc48362465b7-kube-api-access-gw5v8\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193925    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/30699fbf-ab92-43f7-8d71-d4f04fde4381-agent-certs\") pod \"konnectivity-agent-glqxn\" (UID: \"30699fbf-ab92-43f7-8d71-d4f04fde4381\") " pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193953    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0037790b-0ec5-4bd1-ac34-35d830df61d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193978    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.193987    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7e1e854-9d5b-4763-89f1-9829d53da287-ovnkube-script-lib\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194025    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f4d15b6-a854-4feb-82a8-3fc155affffb-dbus\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194048    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-run-systemd\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194063    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-cni-netd\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194109    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-cni-netd\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194159    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194178    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0037790b-0ec5-4bd1-ac34-35d830df61d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194239    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-log-socket\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194182    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194274    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f4d15b6-a854-4feb-82a8-3fc155affffb-dbus\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194285    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-node-log\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194319    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-run-systemd\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194393    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0037790b-0ec5-4bd1-ac34-35d830df61d3-os-release\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194424    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-host-slash\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194482    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7e1e854-9d5b-4763-89f1-9829d53da287-systemd-units\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.194570    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.194615    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.194643    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret podName:4f4d15b6-a854-4feb-82a8-3fc155affffb nodeName:}" failed. No retries permitted until 2026-07-21 04:09:42.694621148 +0000 UTC m=+3.080110049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret") pod "global-pull-secret-syncer-75ljs" (UID: "4f4d15b6-a854-4feb-82a8-3fc155affffb") : object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.194666    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs podName:15bb69f1-6e84-48be-92fb-fc48362465b7 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:42.694650938 +0000 UTC m=+3.080139846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs") pod "network-metrics-daemon-c6kf7" (UID: "15bb69f1-6e84-48be-92fb-fc48362465b7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194867    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/30699fbf-ab92-43f7-8d71-d4f04fde4381-konnectivity-ca\") pod \"konnectivity-agent-glqxn\" (UID: \"30699fbf-ab92-43f7-8d71-d4f04fde4381\") " pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.194981    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0037790b-0ec5-4bd1-ac34-35d830df61d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.195166    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0037790b-0ec5-4bd1-ac34-35d830df61d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.196776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.195727    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7e1e854-9d5b-4763-89f1-9829d53da287-ovnkube-config\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.197209 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.196944    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/30699fbf-ab92-43f7-8d71-d4f04fde4381-agent-certs\") pod \"konnectivity-agent-glqxn\" (UID: \"30699fbf-ab92-43f7-8d71-d4f04fde4381\") " pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:09:42.197209 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.197058    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7e1e854-9d5b-4763-89f1-9829d53da287-ovn-node-metrics-cert\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.199419 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.199405    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 04:09:42.199464 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.199421    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 04:09:42.199464 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.199431    2546 projected.go:196] Error preparing data for projected volume kube-api-access-6px82 for pod openshift-network-diagnostics/network-check-target-xv2m6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:42.199536 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.199481    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82 podName:05f84680-a4fc-4318-b064-6f28c832e6f5 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:42.699470748 +0000 UTC m=+3.084959649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6px82" (UniqueName: "kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82") pod "network-check-target-xv2m6" (UID: "05f84680-a4fc-4318-b064-6f28c832e6f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:42.205292 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.205261    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-922ck\" (UniqueName: \"kubernetes.io/projected/c7e1e854-9d5b-4763-89f1-9829d53da287-kube-api-access-922ck\") pod \"ovnkube-node-xp2zl\" (UID: \"c7e1e854-9d5b-4763-89f1-9829d53da287\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.205508 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.205490    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctpxg\" (UniqueName: \"kubernetes.io/projected/0037790b-0ec5-4bd1-ac34-35d830df61d3-kube-api-access-ctpxg\") pod \"multus-additional-cni-plugins-x5qqx\" (UID: \"0037790b-0ec5-4bd1-ac34-35d830df61d3\") " pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.205564 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.205515    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw5v8\" (UniqueName: \"kubernetes.io/projected/15bb69f1-6e84-48be-92fb-fc48362465b7-kube-api-access-gw5v8\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:42.284648 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.284573    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gjjzt"
Jul 21 04:09:42.296820 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.296792    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f8d25"
Jul 21 04:09:42.305775 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.305754    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph"
Jul 21 04:09:42.311534 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.311517    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r582g"
Jul 21 04:09:42.319400 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.319382    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-82nbl"
Jul 21 04:09:42.326932 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.326915    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vrmch"
Jul 21 04:09:42.335559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.335541    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:09:42.342060 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.342042    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:09:42.348885 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.348871    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x5qqx"
Jul 21 04:09:42.697388 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.697307    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:42.697388 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.697345    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:42.697607 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.697480    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:42.697607 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.697541    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:42.697607 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.697555    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret podName:4f4d15b6-a854-4feb-82a8-3fc155affffb nodeName:}" failed. No retries permitted until 2026-07-21 04:09:43.697536106 +0000 UTC m=+4.083024997 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret") pod "global-pull-secret-syncer-75ljs" (UID: "4f4d15b6-a854-4feb-82a8-3fc155affffb") : object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:42.697607 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.697592    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs podName:15bb69f1-6e84-48be-92fb-fc48362465b7 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:43.697576475 +0000 UTC m=+4.083065368 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs") pod "network-metrics-daemon-c6kf7" (UID: "15bb69f1-6e84-48be-92fb-fc48362465b7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:42.798743 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:42.798711    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:42.798898 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.798884    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 04:09:42.798954 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.798908    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 04:09:42.798954 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.798922    2546 projected.go:196] Error preparing data for projected volume kube-api-access-6px82 for pod openshift-network-diagnostics/network-check-target-xv2m6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:42.799063 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:42.798988    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82 podName:05f84680-a4fc-4318-b064-6f28c832e6f5 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:43.798967965 +0000 UTC m=+4.184456866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6px82" (UniqueName: "kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82") pod "network-check-target-xv2m6" (UID: "05f84680-a4fc-4318-b064-6f28c832e6f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:43.022521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:43.022434    2546 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 04:04:42 +0000 UTC" deadline="2028-02-08 13:31:57.464694922 +0000 UTC"
Jul 21 04:09:43.022521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:43.022470    2546 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13617h22m14.442229608s"
Jul 21 04:09:43.157337 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:43.157307    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:43.157473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:43.157307    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:43.157473 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.157410    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:09:43.157566 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.157487    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:43.705409 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:43.705104    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:43.705409 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:43.705383    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:43.705565 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.705494    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:43.705565 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.705549    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs podName:15bb69f1-6e84-48be-92fb-fc48362465b7 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:45.705531346 +0000 UTC m=+6.091020236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs") pod "network-metrics-daemon-c6kf7" (UID: "15bb69f1-6e84-48be-92fb-fc48362465b7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:43.705938 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.705915    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:43.706019 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.705978    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret podName:4f4d15b6-a854-4feb-82a8-3fc155affffb nodeName:}" failed. No retries permitted until 2026-07-21 04:09:45.705962462 +0000 UTC m=+6.091451356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret") pod "global-pull-secret-syncer-75ljs" (UID: "4f4d15b6-a854-4feb-82a8-3fc155affffb") : object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:43.805855 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:43.805818    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:43.806029 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.806007    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 04:09:43.806090 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.806059    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 04:09:43.806090 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.806074    2546 projected.go:196] Error preparing data for projected volume kube-api-access-6px82 for pod openshift-network-diagnostics/network-check-target-xv2m6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:43.806274 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:43.806144    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82 podName:05f84680-a4fc-4318-b064-6f28c832e6f5 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:45.80610996 +0000 UTC m=+6.191598847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6px82" (UniqueName: "kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82") pod "network-check-target-xv2m6" (UID: "05f84680-a4fc-4318-b064-6f28c832e6f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:44.023186 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.023019    2546 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 04:04:42 +0000 UTC" deadline="2028-02-02 01:46:13.027272811 +0000 UTC"
Jul 21 04:09:44.023186 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.023081    2546 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13461h36m29.004195739s"
Jul 21 04:09:44.157064 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.157021    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:44.157257 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:44.157195    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:09:44.176819 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.176777    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" event={"ID":"0037790b-0ec5-4bd1-ac34-35d830df61d3","Type":"ContainerStarted","Data":"aaf2360acdbb64876ab9c504fafb0c54660e89d18720c1b04397f0c7b6874d17"}
Jul 21 04:09:44.181603 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.181336    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal" event={"ID":"f5f2842cd541864d0773b17e6d2e6e62","Type":"ContainerStarted","Data":"c24ac3e57dc5e75dc853186b6627682a87376450ea57dfbe04902accad5eda03"}
Jul 21 04:09:44.186986 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.186915    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-glqxn" event={"ID":"30699fbf-ab92-43f7-8d71-d4f04fde4381","Type":"ContainerStarted","Data":"8feef7964c49f56107cd79665195a46b82fc3d07aa465ccd13675543475c6a66"}
Jul 21 04:09:44.193978 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.193950    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerStarted","Data":"49dfc1ef80600fc2d08b351785a2c46e4fb4daf383232cd067cbeb19177a693e"}
Jul 21 04:09:44.199042 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.199016    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-82nbl" event={"ID":"2140b0ce-d757-4b6d-b07a-82e3b8e15f90","Type":"ContainerStarted","Data":"dbbe45c9669dd3553477f9087730dfbbb6aee8a29c2b5c86e6d5c11712429bd0"}
Jul 21 04:09:44.212878 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.212776    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vrmch" event={"ID":"b761d554-5f1b-4da7-b197-d4cd24446e11","Type":"ContainerStarted","Data":"2fe205d079b3d3ecbda9ddac8063432e73a3f18edd39d2b3bb20515653f4246e"}
Jul 21 04:09:44.225036 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.224939    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph" event={"ID":"b50b377c-8166-4fe6-9c61-e739659c12c7","Type":"ContainerStarted","Data":"02a4c9174575729015bd3eae46d5b13fb802f9a91574a67a3d5baa8a27230b4e"}
Jul 21 04:09:44.232262 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.232235    2546 generic.go:356] "Generic (PLEG): container finished" podID="3908927fc6f60eab4bf00fa616e3360a" containerID="fd851199a93d8ee004a3a55bdd938d845b0e7c211db7d9046b179661f606d90b" exitCode=0
Jul 21 04:09:44.232392 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.232313    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal" event={"ID":"3908927fc6f60eab4bf00fa616e3360a","Type":"ContainerDied","Data":"fd851199a93d8ee004a3a55bdd938d845b0e7c211db7d9046b179661f606d90b"}
Jul 21 04:09:44.251418 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.251373    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f8d25" event={"ID":"78a71799-4b84-44a2-a968-2b591c4f785d","Type":"ContainerStarted","Data":"60235742d37b91cb59b26dd6de965ece417cbe303374c0241217c50358204d80"}
Jul 21 04:09:44.254256 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.254206    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal" podStartSLOduration=3.254168434 podStartE2EDuration="3.254168434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:09:44.198610203 +0000 UTC m=+4.584099186" watchObservedRunningTime="2026-07-21 04:09:44.254168434 +0000 UTC m=+4.639657349"
Jul 21 04:09:44.264065 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.263885    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gjjzt" event={"ID":"3798e436-1a3e-47a4-8911-724c05dc7e53","Type":"ContainerStarted","Data":"a3a458298d4349e0850b43b25d49777bccb504cf1896d21242140e940388ab9a"}
Jul 21 04:09:44.269790 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:44.269764    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r582g" event={"ID":"d72963f6-4559-479b-b8d1-633a1cab9d0d","Type":"ContainerStarted","Data":"3c002aaf4c2c38537911a6a2408095843b1dadf368481646bd289192112897ac"}
Jul 21 04:09:45.156608 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:45.156568    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:45.157084 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.156706    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:45.157350 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:45.157330    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:45.157463 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.157442    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:09:45.295338 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:45.295303    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal" event={"ID":"3908927fc6f60eab4bf00fa616e3360a","Type":"ContainerStarted","Data":"b3f47e2cbdb7f81ce9e383fdc6c1f90c585135144955e95ca8ad599afe883677"}
Jul 21 04:09:45.721318 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:45.721279    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:45.721504 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:45.721333    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:45.721504 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.721497    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:45.721611 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.721558    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs podName:15bb69f1-6e84-48be-92fb-fc48362465b7 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:49.72153972 +0000 UTC m=+10.107028611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs") pod "network-metrics-daemon-c6kf7" (UID: "15bb69f1-6e84-48be-92fb-fc48362465b7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:45.722045 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.721943    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:45.722045 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.721993    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret podName:4f4d15b6-a854-4feb-82a8-3fc155affffb nodeName:}" failed. No retries permitted until 2026-07-21 04:09:49.721978652 +0000 UTC m=+10.107467553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret") pod "global-pull-secret-syncer-75ljs" (UID: "4f4d15b6-a854-4feb-82a8-3fc155affffb") : object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:45.822073 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:45.822031    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:45.822283 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.822266    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 04:09:45.822345 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.822288    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 04:09:45.822345 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.822301    2546 projected.go:196] Error preparing data for projected volume kube-api-access-6px82 for pod openshift-network-diagnostics/network-check-target-xv2m6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:45.822450 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:45.822352    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82 podName:05f84680-a4fc-4318-b064-6f28c832e6f5 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:49.822335022 +0000 UTC m=+10.207823913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6px82" (UniqueName: "kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82") pod "network-check-target-xv2m6" (UID: "05f84680-a4fc-4318-b064-6f28c832e6f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:46.160177 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:46.159599    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:46.160177 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:46.159735    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:09:47.156792 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:47.156762    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:47.156985 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:47.156863    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:47.156985 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:47.156906    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:47.156985 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:47.156952    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:09:48.158263 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:48.158228    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:48.158750 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:48.158385    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:09:49.156620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:49.156587    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:49.156802 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:49.156601    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:49.156802 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.156722    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:49.156885 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.156791    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:09:49.756367 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:49.756325    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:49.756789 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:49.756379    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:49.756789 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.756536    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:49.756789 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.756604    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs podName:15bb69f1-6e84-48be-92fb-fc48362465b7 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:57.756583876 +0000 UTC m=+18.142072769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs") pod "network-metrics-daemon-c6kf7" (UID: "15bb69f1-6e84-48be-92fb-fc48362465b7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:49.756992 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.756976    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:49.757042 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.757025    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret podName:4f4d15b6-a854-4feb-82a8-3fc155affffb nodeName:}" failed. No retries permitted until 2026-07-21 04:09:57.757011291 +0000 UTC m=+18.142500182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret") pod "global-pull-secret-syncer-75ljs" (UID: "4f4d15b6-a854-4feb-82a8-3fc155affffb") : object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:49.857394 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:49.857352    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:49.857588 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.857520    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 04:09:49.857588 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.857545    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 04:09:49.857588 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.857559    2546 projected.go:196] Error preparing data for projected volume kube-api-access-6px82 for pod openshift-network-diagnostics/network-check-target-xv2m6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:49.857747 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:49.857643    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82 podName:05f84680-a4fc-4318-b064-6f28c832e6f5 nodeName:}" failed. No retries permitted until 2026-07-21 04:09:57.857622905 +0000 UTC m=+18.243111795 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6px82" (UniqueName: "kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82") pod "network-check-target-xv2m6" (UID: "05f84680-a4fc-4318-b064-6f28c832e6f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:50.158827 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:50.158376    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:50.158827 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:50.158476    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:09:51.157731 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:51.157157    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:51.157731 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:51.157315    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:09:51.157731 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:51.157157    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:51.157731 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:51.157419    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:52.158367 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:52.158323    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:52.158839 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:52.158474    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:09:53.157449 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:53.157410    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:53.157631 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:53.157535    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:53.157631 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:53.157599    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:53.157778 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:53.157744    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:09:54.157986 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:54.157947    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:54.158435 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:54.158101    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:09:55.156946 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:55.156907    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:55.157135 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:55.157052    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:09:55.157135 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:55.157073    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:55.157274 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:55.157206    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:56.157392 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:56.157359    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:56.157808 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:56.157499    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:09:57.157301 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:57.157271    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:57.157487 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:57.157308    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:57.157487 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.157377    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:57.157487 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.157462    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:09:57.819830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:57.819781    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:57.819830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:57.819836    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:57.820047 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.819954    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:57.820047 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.819954    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:57.820047 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.820024    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs podName:15bb69f1-6e84-48be-92fb-fc48362465b7 nodeName:}" failed. No retries permitted until 2026-07-21 04:10:13.820004815 +0000 UTC m=+34.205493709 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs") pod "network-metrics-daemon-c6kf7" (UID: "15bb69f1-6e84-48be-92fb-fc48362465b7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:09:57.820047 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.820043    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret podName:4f4d15b6-a854-4feb-82a8-3fc155affffb nodeName:}" failed. No retries permitted until 2026-07-21 04:10:13.820032134 +0000 UTC m=+34.205521027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret") pod "global-pull-secret-syncer-75ljs" (UID: "4f4d15b6-a854-4feb-82a8-3fc155affffb") : object "kube-system"/"original-pull-secret" not registered
Jul 21 04:09:57.920720 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:57.920686    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:57.920880 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.920824    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 04:09:57.920880 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.920842    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 04:09:57.920880 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.920852    2546 projected.go:196] Error preparing data for projected volume kube-api-access-6px82 for pod openshift-network-diagnostics/network-check-target-xv2m6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:57.921008 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:57.920899    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82 podName:05f84680-a4fc-4318-b064-6f28c832e6f5 nodeName:}" failed. No retries permitted until 2026-07-21 04:10:13.920886455 +0000 UTC m=+34.306375342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6px82" (UniqueName: "kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82") pod "network-check-target-xv2m6" (UID: "05f84680-a4fc-4318-b064-6f28c832e6f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:09:58.157369 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:58.157295    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:09:58.157519 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:58.157431    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:09:59.156596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:59.156566    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:09:59.156778 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:09:59.156571    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:09:59.156778 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:59.156676    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:09:59.156923 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:09:59.156774    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:00.157302 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:00.157267    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:00.157720 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:00.157423    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:01.156580 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:01.156543    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:01.156778 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:01.156681    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:01.157090 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:01.157064    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:01.157238 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:01.157164    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:02.157197 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.156790    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:02.157197 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:02.157021    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:02.328784 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.328709    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-glqxn" event={"ID":"30699fbf-ab92-43f7-8d71-d4f04fde4381","Type":"ContainerStarted","Data":"55e9b5aa1a2aa26e403e8da0c1f29dfd6491ee1adab79f5bfe884799dfff5f79"}
Jul 21 04:10:02.330076 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.330048    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-82nbl" event={"ID":"2140b0ce-d757-4b6d-b07a-82e3b8e15f90","Type":"ContainerStarted","Data":"57d522a280a91d888eed01e493b2ad6d293a8a626851b67c80bedcfef93c8082"}
Jul 21 04:10:02.331272 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.331253    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vrmch" event={"ID":"b761d554-5f1b-4da7-b197-d4cd24446e11","Type":"ContainerStarted","Data":"cf6e65544b95997de294d68a805690e1f6e27ba678ca825616fe6fcccf9f6685"}
Jul 21 04:10:02.332586 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.332566    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph" event={"ID":"b50b377c-8166-4fe6-9c61-e739659c12c7","Type":"ContainerStarted","Data":"5cdb888d586128e4b36f135e26b72a6e40d5354de0155b48fe4c2920a86a4451"}
Jul 21 04:10:02.333900 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.333880    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gjjzt" event={"ID":"3798e436-1a3e-47a4-8911-724c05dc7e53","Type":"ContainerStarted","Data":"64b11fa63638dd02a5e07c8279529ee96bbafd0463d36ee9ac91684183b859ed"}
Jul 21 04:10:02.335194 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.335159    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r582g" event={"ID":"d72963f6-4559-479b-b8d1-633a1cab9d0d","Type":"ContainerStarted","Data":"972f12f9a9949ecc409074f7d9b06460e654af1cf133526c744531428ed733ec"}
Jul 21 04:10:02.336417 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.336400    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" event={"ID":"0037790b-0ec5-4bd1-ac34-35d830df61d3","Type":"ContainerStarted","Data":"f7c06e3d1cf362e9ec527c07ce46a1a432c96b4725aa3460cc6e84272f22aed7"}
Jul 21 04:10:02.345056 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.345019    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-glqxn" podStartSLOduration=8.204232887 podStartE2EDuration="22.345002892s" totalImagesPullingTime="14.140770005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:02.344930416 +0000 UTC m=+22.730419324" watchObservedRunningTime="2026-07-21 04:10:02.345002892 +0000 UTC m=+22.730491802"
Jul 21 04:10:02.345486 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.345462    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal" podStartSLOduration=21.345453628 podStartE2EDuration="21.345453628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:09:45.312956309 +0000 UTC m=+5.698445219" watchObservedRunningTime="2026-07-21 04:10:02.345453628 +0000 UTC m=+22.730942536"
Jul 21 04:10:02.360534 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.360490    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vrmch" podStartSLOduration=8.212225008 podStartE2EDuration="22.360477228s" totalImagesPullingTime="14.14825222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:02.360069996 +0000 UTC m=+22.745558907" watchObservedRunningTime="2026-07-21 04:10:02.360477228 +0000 UTC m=+22.745966137"
Jul 21 04:10:02.398937 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.398893    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gjjzt" podStartSLOduration=3.586645086 podStartE2EDuration="22.398877631s" totalImagesPullingTime="18.812232545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:02.398856917 +0000 UTC m=+22.784345826" watchObservedRunningTime="2026-07-21 04:10:02.398877631 +0000 UTC m=+22.784366540"
Jul 21 04:10:02.420973 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.420924    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r582g" podStartSLOduration=3.603099233 podStartE2EDuration="22.420904223s" totalImagesPullingTime="18.81780499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:02.420863832 +0000 UTC m=+22.806352743" watchObservedRunningTime="2026-07-21 04:10:02.420904223 +0000 UTC m=+22.806393136"
Jul 21 04:10:02.436475 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.436427    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-82nbl" podStartSLOduration=3.902763415 podStartE2EDuration="22.436410875s" totalImagesPullingTime="18.53364746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:02.43593416 +0000 UTC m=+22.821423069" watchObservedRunningTime="2026-07-21 04:10:02.436410875 +0000 UTC m=+22.821899784"
Jul 21 04:10:02.438268 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.438242    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:02.532912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.532882    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:02.625747 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:02.625712    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:03.157360 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.157102    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:03.157945 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.157169    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:03.157945 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:03.157401    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:03.157945 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:03.157516    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:03.339577 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.339543    2546 generic.go:356] "Generic (PLEG): container finished" podID="0037790b-0ec5-4bd1-ac34-35d830df61d3" containerID="f7c06e3d1cf362e9ec527c07ce46a1a432c96b4725aa3460cc6e84272f22aed7" exitCode=0
Jul 21 04:10:03.339730 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.339634    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" event={"ID":"0037790b-0ec5-4bd1-ac34-35d830df61d3","Type":"ContainerDied","Data":"f7c06e3d1cf362e9ec527c07ce46a1a432c96b4725aa3460cc6e84272f22aed7"}
Jul 21 04:10:03.342508 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.342484    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:10:03.342861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.342839    2546 generic.go:356] "Generic (PLEG): container finished" podID="c7e1e854-9d5b-4763-89f1-9829d53da287" containerID="ba227c567a32660249da20ca58960527c170ed1463c3cfc6b1a73865844969e3" exitCode=1
Jul 21 04:10:03.342965 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.342910    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerStarted","Data":"1af354716023dcb3b4c1cd7eb0338741843cdf25b4efb2316799b1f5959c0d6a"}
Jul 21 04:10:03.343033 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.343018    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerStarted","Data":"6622052f2806af2d22f9e5058f9cc6926087b374a871267399e1d2c3ad1135b5"}
Jul 21 04:10:03.343095 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.343076    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerStarted","Data":"7dffa124ae80f18e06c473811c32ee3a26fd94e683cde149577f6548adf4932b"}
Jul 21 04:10:03.343173 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.343153    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerStarted","Data":"5aedd197a8bfea9c0a14454c4717fe735938feaa6445f7af4044b0faaf00999f"}
Jul 21 04:10:03.343237 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.343185    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerDied","Data":"ba227c567a32660249da20ca58960527c170ed1463c3cfc6b1a73865844969e3"}
Jul 21 04:10:03.343237 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.343226    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerStarted","Data":"1b6061796de0698e080f957270b3e0e907730d2808b39b16007af18d0398f868"}
Jul 21 04:10:03.347786 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.347760    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f8d25" event={"ID":"78a71799-4b84-44a2-a968-2b591c4f785d","Type":"ContainerStarted","Data":"c69d1402cd17cbf7c9c69234a80f8e7d83035519ab811adf2249c94d8e2f4e90"}
Jul 21 04:10:03.376987 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.376941    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f8d25" podStartSLOduration=4.692434665 podStartE2EDuration="23.376927522s" totalImagesPullingTime="18.684492857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:03.376752314 +0000 UTC m=+23.762241223" watchObservedRunningTime="2026-07-21 04:10:03.376927522 +0000 UTC m=+23.762416431"
Jul 21 04:10:03.496786 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.496691    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:03.613838 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.613815    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:03.672235 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.672215    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vrmch_b761d554-5f1b-4da7-b197-d4cd24446e11/node-ca/0.log"
Jul 21 04:10:03.744274 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.744249    2546 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 04:10:03.791813 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.791781    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:03.890783 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:03.890756    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:04.051562 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:04.051328    2546 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T04:10:03.744266773Z","UUID":"d5589315-fe3c-4018-91c9-471031ee4eb3","Handler":null,"Name":"","Endpoint":""}
Jul 21 04:10:04.054247 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:04.054223    2546 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 04:10:04.054396 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:04.054270    2546 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 04:10:04.156803 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:04.156766    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:04.156964 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:04.156907    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:04.352752 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:04.352669    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph" event={"ID":"b50b377c-8166-4fe6-9c61-e739659c12c7","Type":"ContainerStarted","Data":"43f3003ba5d312179018967b05162186567aac16264f38daaf834ca73d499a8e"}
Jul 21 04:10:04.725485 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:04.725455    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:04.962916 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:04.962884    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:10:04.963454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:04.963434    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:10:05.156492 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:05.156474    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:05.156612 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:05.156582    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:05.156681 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:05.156615    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:05.156734 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:05.156696    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:05.359053 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:05.358815    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:10:05.359519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:05.359493    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerStarted","Data":"4f11cbbc3950824de5ca4803bded3d2b16b3935593c80f19546907e00ccf4264"}
Jul 21 04:10:05.362059 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:05.362032    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph" event={"ID":"b50b377c-8166-4fe6-9c61-e739659c12c7","Type":"ContainerStarted","Data":"3be76c8a629efcf74c8d3b8771aecca37e5a4b6f720e5106de9576e1fc594b54"}
Jul 21 04:10:05.364576 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:05.362432    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:10:05.364576 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:05.363692    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-glqxn"
Jul 21 04:10:05.385092 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:05.385044    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh6ph" podStartSLOduration=3.891454516 podStartE2EDuration="25.38502958s" totalImagesPullingTime="21.493575064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:05.384586901 +0000 UTC m=+25.770075819" watchObservedRunningTime="2026-07-21 04:10:05.38502958 +0000 UTC m=+25.770518489"
Jul 21 04:10:06.156988 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:06.156945    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:06.157203 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:06.157103    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:07.156989 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:07.156956    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:07.157596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:07.156957    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:07.157596 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:07.157078    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:07.157596 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:07.157151    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:07.842042 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:07.841848    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:08.157411 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.157326    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:08.157987 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:08.157474    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:08.371273 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.371238    2546 generic.go:356] "Generic (PLEG): container finished" podID="0037790b-0ec5-4bd1-ac34-35d830df61d3" containerID="2769a1bb629bdb371bdd18d7d87056380883ebd56928ae88011027b1eb4801cc" exitCode=0
Jul 21 04:10:08.371412 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.371323    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" event={"ID":"0037790b-0ec5-4bd1-ac34-35d830df61d3","Type":"ContainerDied","Data":"2769a1bb629bdb371bdd18d7d87056380883ebd56928ae88011027b1eb4801cc"}
Jul 21 04:10:08.374628 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.374612    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:10:08.374988 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.374966    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerStarted","Data":"d94b75aaf22d17a13e7b3a1c9828a83970e2790d10e536d27f358ce792766406"}
Jul 21 04:10:08.375248 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.375228    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:10:08.375333 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.375284    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:10:08.375424 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.375407    2546 scope.go:117] "RemoveContainer" containerID="ba227c567a32660249da20ca58960527c170ed1463c3cfc6b1a73865844969e3"
Jul 21 04:10:08.392488 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.392463    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:10:08.904914 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:08.904885    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:09.095796 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.095765    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:09.156969 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.156886    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:09.157091 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.156886    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:09.157091 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:09.157015    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:09.157091 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:09.157045    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:09.280361 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.280331    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:09.337038 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.337015    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:09.378800 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.378766    2546 generic.go:356] "Generic (PLEG): container finished" podID="0037790b-0ec5-4bd1-ac34-35d830df61d3" containerID="8a63ff36f67b84d14255e2833e0e3b30861a80804dcc82fd370e7dafa39687cb" exitCode=0
Jul 21 04:10:09.378935 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.378800    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" event={"ID":"0037790b-0ec5-4bd1-ac34-35d830df61d3","Type":"ContainerDied","Data":"8a63ff36f67b84d14255e2833e0e3b30861a80804dcc82fd370e7dafa39687cb"}
Jul 21 04:10:09.382815 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.382792    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:10:09.383161 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.383137    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" event={"ID":"c7e1e854-9d5b-4763-89f1-9829d53da287","Type":"ContainerStarted","Data":"4d4d879384397f9493a55e635407d434d3924755ddb39f020c8a37ca35f1595a"}
Jul 21 04:10:09.383416 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.383395    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:10:09.398955 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.398935    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:10:09.408068 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.408002    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:09.431423 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.431379    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl" podStartSLOduration=10.162794852 podStartE2EDuration="29.431359631s" totalImagesPullingTime="19.268564779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:09.429976361 +0000 UTC m=+29.815465270" watchObservedRunningTime="2026-07-21 04:10:09.431359631 +0000 UTC m=+29.816848521"
Jul 21 04:10:09.511071 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.511046    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:09.610747 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.610717    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:09.801388 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:09.801359    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:10.158137 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:10.158042    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:10.158273 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:10.158194    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:10.387984 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:10.387951    2546 generic.go:356] "Generic (PLEG): container finished" podID="0037790b-0ec5-4bd1-ac34-35d830df61d3" containerID="7fe8f0f2cb8b245e917604690ab91f729d1fa675decc23087c4cba64935b3b94" exitCode=0
Jul 21 04:10:10.388327 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:10.388042    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" event={"ID":"0037790b-0ec5-4bd1-ac34-35d830df61d3","Type":"ContainerDied","Data":"7fe8f0f2cb8b245e917604690ab91f729d1fa675decc23087c4cba64935b3b94"}
Jul 21 04:10:10.796559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:10.796520    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:11.084065 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:11.083954    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:11.156821 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:11.156790    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:11.156970 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:11.156791    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:11.156970 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:11.156897    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:11.157044 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:11.156969    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:11.195239 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:11.195190    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:11.402963 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:11.402884    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:11.606310 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:11.606270    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:11.797774 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:11.797705    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:12.157259 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:12.157183    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:12.157401 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:12.157319    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:12.731516 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:12.731473    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:12.833344 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:12.833287    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:13.000499 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.000415    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:13.157178 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.157146    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:13.157178 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.157187    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:13.157420 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.157303    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:13.157473 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.157444    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:13.201327 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.201292    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:13.399863 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.399772    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:13.599398 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.599369    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:13.658052 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.657965    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:13.804769 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.804694    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:13.838804 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.838768    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:13.838974 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.838814    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:13.838974 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.838906    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 04:10:13.838974 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.838915    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:10:13.839137 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.838979    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs podName:15bb69f1-6e84-48be-92fb-fc48362465b7 nodeName:}" failed. No retries permitted until 2026-07-21 04:10:45.838961583 +0000 UTC m=+66.224450470 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs") pod "network-metrics-daemon-c6kf7" (UID: "15bb69f1-6e84-48be-92fb-fc48362465b7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 04:10:13.839137 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.838995    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret podName:4f4d15b6-a854-4feb-82a8-3fc155affffb nodeName:}" failed. No retries permitted until 2026-07-21 04:10:45.838988135 +0000 UTC m=+66.224477022 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret") pod "global-pull-secret-syncer-75ljs" (UID: "4f4d15b6-a854-4feb-82a8-3fc155affffb") : object "kube-system"/"original-pull-secret" not registered
Jul 21 04:10:13.940217 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:13.940138    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:13.940372 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.940313    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 04:10:13.940372 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.940329    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 04:10:13.940372 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.940338    2546 projected.go:196] Error preparing data for projected volume kube-api-access-6px82 for pod openshift-network-diagnostics/network-check-target-xv2m6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:10:13.940496 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:13.940395    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82 podName:05f84680-a4fc-4318-b064-6f28c832e6f5 nodeName:}" failed. No retries permitted until 2026-07-21 04:10:45.940377639 +0000 UTC m=+66.325866549 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6px82" (UniqueName: "kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82") pod "network-check-target-xv2m6" (UID: "05f84680-a4fc-4318-b064-6f28c832e6f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 04:10:14.157185 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:14.157153    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:14.157496 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:14.157303    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:14.415653 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:14.415618    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:14.471016 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:14.470989    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:15.000647 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.000619    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:15.157250 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.157219    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:15.157415 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.157224    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:15.157415 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:15.157312    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:15.157415 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:15.157401    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:15.193876 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.193851    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:15.398414 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.398332    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:15.401868 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.401838    2546 generic.go:356] "Generic (PLEG): container finished" podID="0037790b-0ec5-4bd1-ac34-35d830df61d3" containerID="a6c0635f8cc7b80b0b5c123c3d87eeedffb290c0e2250c3eee6d88a89f560b7d" exitCode=0
Jul 21 04:10:15.401980 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.401877    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" event={"ID":"0037790b-0ec5-4bd1-ac34-35d830df61d3","Type":"ContainerDied","Data":"a6c0635f8cc7b80b0b5c123c3d87eeedffb290c0e2250c3eee6d88a89f560b7d"}
Jul 21 04:10:15.455416 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.455381    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:15.618008 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.617951    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:15.798774 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:15.798729    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:16.157361 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:16.157284    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:16.157703 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:16.157399    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:16.406763 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:16.406734    2546 generic.go:356] "Generic (PLEG): container finished" podID="0037790b-0ec5-4bd1-ac34-35d830df61d3" containerID="2c1bdf02353e89783c0b6afcee7f13544728750aef5f8a04db8b5dd28593ab73" exitCode=0
Jul 21 04:10:16.406912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:16.406783    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" event={"ID":"0037790b-0ec5-4bd1-ac34-35d830df61d3","Type":"ContainerDied","Data":"2c1bdf02353e89783c0b6afcee7f13544728750aef5f8a04db8b5dd28593ab73"}
Jul 21 04:10:16.796378 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:16.796182    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:17.156947 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:17.156872    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:17.157094 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:17.156962    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:17.157094 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:17.157016    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:17.157184 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:17.157096    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:17.197325 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:17.197296    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:17.396129 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:17.396081    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:17.412153 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:17.412066    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" event={"ID":"0037790b-0ec5-4bd1-ac34-35d830df61d3","Type":"ContainerStarted","Data":"5876fabbb9af19f329ccc1492dcef90c8dfe05f3fb983f165c8e28429fdbe7d4"}
Jul 21 04:10:17.436219 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:17.436180    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x5qqx" podStartSLOduration=8.903607835 podStartE2EDuration="37.436166044s" totalImagesPullingTime="28.532558209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:17.434997772 +0000 UTC m=+37.820486681" watchObservedRunningTime="2026-07-21 04:10:17.436166044 +0000 UTC m=+37.821654952"
Jul 21 04:10:17.600385 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:17.600352    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:17.797281 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:17.797251    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:18.157076 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:18.156991    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:18.157248 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:18.157091    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:18.606982 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:18.606949    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:18.800784 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:18.800739    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:19.157354 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:19.157325    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:19.157532 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:19.157327    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:19.157532 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:19.157428    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:19.157532 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:19.157512    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:19.201277 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:19.201239    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:19.600430 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:19.600400    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:19.800358 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:19.800327    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:20.157758 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:20.157731    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:20.157911 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:20.157815    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:21.157157 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:21.157101    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:21.157533 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:21.157221    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:21.157533 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:21.157275    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:21.157533 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:21.157354    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:22.157347 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:22.157311    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:22.157711 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:22.157449    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:23.157550 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:23.157516    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:23.157943 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:23.157621    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:23.157943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:23.157672    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:23.157943 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:23.157757    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:24.157090 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:24.157060    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:24.157267 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:24.157182    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:24.452641 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:24.452556    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:24.540581 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:24.540555    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:24.641462 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:24.641434    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:24.741594 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:24.741564    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:24.839083 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:24.839049    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:25.156612 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.156538    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:25.156767 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:25.156661    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:25.156767 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.156712    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:25.156852 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:25.156827    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:25.522706 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.522678    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:25.700589 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.700557    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-75ljs"]
Jul 21 04:10:25.700736 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.700718    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:25.700851 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:25.700828    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:25.702987 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.702957    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c6kf7"]
Jul 21 04:10:25.703100 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.703076    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:25.703239 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:25.703217    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:25.705894 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.705869    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xv2m6"]
Jul 21 04:10:25.706014 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.705975    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:25.706173 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:25.706061    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:25.873776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.873697    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:25.978034 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:25.978000    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:27.156877 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:27.156847    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:27.157481 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:27.156850    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:27.157481 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:27.156975    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:27.157481 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:27.156857    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:27.157481 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:27.157035    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:27.157481 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:27.157099    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:29.156817 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.156786    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:29.157204 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.156786    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:29.157204 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:29.156891    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xv2m6" podUID="05f84680-a4fc-4318-b064-6f28c832e6f5"
Jul 21 04:10:29.157204 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.156786    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:29.157204 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:29.156954    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75ljs" podUID="4f4d15b6-a854-4feb-82a8-3fc155affffb"
Jul 21 04:10:29.157204 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:29.157032    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6kf7" podUID="15bb69f1-6e84-48be-92fb-fc48362465b7"
Jul 21 04:10:29.446281 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.446188    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:29.546128 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.546085    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:29.926984 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.926954    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeReady"
Jul 21 04:10:29.927157 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.927062    2546 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 04:10:29.973302 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.973265    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-99xh7"]
Jul 21 04:10:29.987478 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.987452    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hnbk7"]
Jul 21 04:10:29.987646 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.987577    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-99xh7"
Jul 21 04:10:29.990340 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.990154    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 04:10:29.990468 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.990174    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 04:10:29.990468 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.990207    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zksv2\""
Jul 21 04:10:29.990468 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:29.990274    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 04:10:30.000245 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.000223    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-99xh7"]
Jul 21 04:10:30.000327 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.000257    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnbk7"]
Jul 21 04:10:30.000327 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.000319    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.003091 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.003071    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t87r9\""
Jul 21 04:10:30.003091 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.003072    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 04:10:30.003298 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.003072    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 04:10:30.003298 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.003235    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 04:10:30.003405 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.003343    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 04:10:30.075221 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.075185    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qkv8k"]
Jul 21 04:10:30.093424 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.093401    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qkv8k"]
Jul 21 04:10:30.093559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.093513    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.096227 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.096192    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 04:10:30.096227 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.096222    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 04:10:30.096418 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.096312    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b7grb\""
Jul 21 04:10:30.158552 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.158520    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzbqb\" (UniqueName: \"kubernetes.io/projected/e2b54ec0-f78b-4aa6-a64f-041017187539-kube-api-access-fzbqb\") pod \"ingress-canary-99xh7\" (UID: \"e2b54ec0-f78b-4aa6-a64f-041017187539\") " pod="openshift-ingress-canary/ingress-canary-99xh7"
Jul 21 04:10:30.158902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.158570    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.158902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.158608    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2b54ec0-f78b-4aa6-a64f-041017187539-cert\") pod \"ingress-canary-99xh7\" (UID: \"e2b54ec0-f78b-4aa6-a64f-041017187539\") " pod="openshift-ingress-canary/ingress-canary-99xh7"
Jul 21 04:10:30.158902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.158629    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qksxb\" (UniqueName: \"kubernetes.io/projected/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-kube-api-access-qksxb\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.158902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.158649    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-crio-socket\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.158902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.158714    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-data-volume\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.158902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.158767    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.175588 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.175544    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:30.259640 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259612    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-crio-socket\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.259812 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259647    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2915ad8b-22a5-489c-ba33-3f592a81241a-tmp-dir\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.259812 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259679    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-data-volume\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.259812 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259698    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2915ad8b-22a5-489c-ba33-3f592a81241a-config-volume\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.259812 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259726    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.259812 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259795    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzbqb\" (UniqueName: \"kubernetes.io/projected/e2b54ec0-f78b-4aa6-a64f-041017187539-kube-api-access-fzbqb\") pod \"ingress-canary-99xh7\" (UID: \"e2b54ec0-f78b-4aa6-a64f-041017187539\") " pod="openshift-ingress-canary/ingress-canary-99xh7"
Jul 21 04:10:30.260104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259820    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-crio-socket\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.260104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259887    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.260104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259931    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2b54ec0-f78b-4aa6-a64f-041017187539-cert\") pod \"ingress-canary-99xh7\" (UID: \"e2b54ec0-f78b-4aa6-a64f-041017187539\") " pod="openshift-ingress-canary/ingress-canary-99xh7"
Jul 21 04:10:30.260104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.259963    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldgfb\" (UniqueName: \"kubernetes.io/projected/2915ad8b-22a5-489c-ba33-3f592a81241a-kube-api-access-ldgfb\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.260104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.260006    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-data-volume\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.260104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.260048    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qksxb\" (UniqueName: \"kubernetes.io/projected/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-kube-api-access-qksxb\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.260104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.260068    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2915ad8b-22a5-489c-ba33-3f592a81241a-metrics-tls\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.260493 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.260475    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.264065 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.264046    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.264179 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.264142    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2b54ec0-f78b-4aa6-a64f-041017187539-cert\") pod \"ingress-canary-99xh7\" (UID: \"e2b54ec0-f78b-4aa6-a64f-041017187539\") " pod="openshift-ingress-canary/ingress-canary-99xh7"
Jul 21 04:10:30.268867 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.268836    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qksxb\" (UniqueName: \"kubernetes.io/projected/e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314-kube-api-access-qksxb\") pod \"insights-runtime-extractor-hnbk7\" (UID: \"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314\") " pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.269094 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.269078    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzbqb\" (UniqueName: \"kubernetes.io/projected/e2b54ec0-f78b-4aa6-a64f-041017187539-kube-api-access-fzbqb\") pod \"ingress-canary-99xh7\" (UID: \"e2b54ec0-f78b-4aa6-a64f-041017187539\") " pod="openshift-ingress-canary/ingress-canary-99xh7"
Jul 21 04:10:30.297977 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.297950    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-99xh7"
Jul 21 04:10:30.308157 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.308129    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnbk7"
Jul 21 04:10:30.360552 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.360519    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2915ad8b-22a5-489c-ba33-3f592a81241a-tmp-dir\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.360732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.360573    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2915ad8b-22a5-489c-ba33-3f592a81241a-config-volume\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.360732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.360640    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldgfb\" (UniqueName: \"kubernetes.io/projected/2915ad8b-22a5-489c-ba33-3f592a81241a-kube-api-access-ldgfb\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.360732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.360670    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2915ad8b-22a5-489c-ba33-3f592a81241a-metrics-tls\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.361644 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.361442    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2915ad8b-22a5-489c-ba33-3f592a81241a-config-volume\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.361644 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.361613    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2915ad8b-22a5-489c-ba33-3f592a81241a-tmp-dir\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.363382 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.363353    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2915ad8b-22a5-489c-ba33-3f592a81241a-metrics-tls\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.371417 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.371370    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldgfb\" (UniqueName: \"kubernetes.io/projected/2915ad8b-22a5-489c-ba33-3f592a81241a-kube-api-access-ldgfb\") pod \"dns-default-qkv8k\" (UID: \"2915ad8b-22a5-489c-ba33-3f592a81241a\") " pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.402617 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.402332    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:30.494732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.494396    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-99xh7"]
Jul 21 04:10:30.498575 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.498550    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnbk7"]
Jul 21 04:10:30.500467 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:30.500438    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b54ec0_f78b_4aa6_a64f_041017187539.slice/crio-0dc4ceabba327a5d78d71a8aafd84b447b74892675407755d1a89a4b68ced0c4 WatchSource:0}: Error finding container 0dc4ceabba327a5d78d71a8aafd84b447b74892675407755d1a89a4b68ced0c4: Status 404 returned error can't find the container with id 0dc4ceabba327a5d78d71a8aafd84b447b74892675407755d1a89a4b68ced0c4
Jul 21 04:10:30.500669 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:30.500650    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d40ba1_7f18_42ee_a5d2_b4c2b1f69314.slice/crio-907ad46c32fe221593579c4deddeda768ae5282f3330068c382cb29fc7c93af2 WatchSource:0}: Error finding container 907ad46c32fe221593579c4deddeda768ae5282f3330068c382cb29fc7c93af2: Status 404 returned error can't find the container with id 907ad46c32fe221593579c4deddeda768ae5282f3330068c382cb29fc7c93af2
Jul 21 04:10:30.570721 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:30.568303    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qkv8k"]
Jul 21 04:10:30.572313 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:30.572280    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2915ad8b_22a5_489c_ba33_3f592a81241a.slice/crio-d498ad1bcf64384dda61482692883535db1bc84b66d39f7d8e9ab98df385cf80 WatchSource:0}: Error finding container d498ad1bcf64384dda61482692883535db1bc84b66d39f7d8e9ab98df385cf80: Status 404 returned error can't find the container with id d498ad1bcf64384dda61482692883535db1bc84b66d39f7d8e9ab98df385cf80
Jul 21 04:10:31.157145 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.157095    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:31.157318 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.157153    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:31.157318 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.157095    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:31.161511 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.161481    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 04:10:31.161864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.161528    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 04:10:31.161864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.161701    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 04:10:31.161864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.161781    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 04:10:31.162284 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.162015    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qwqtb\""
Jul 21 04:10:31.162284 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.162145    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9v8hm\""
Jul 21 04:10:31.447475 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.447395    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qkv8k" event={"ID":"2915ad8b-22a5-489c-ba33-3f592a81241a","Type":"ContainerStarted","Data":"d498ad1bcf64384dda61482692883535db1bc84b66d39f7d8e9ab98df385cf80"}
Jul 21 04:10:31.449170 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.449144    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnbk7" event={"ID":"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314","Type":"ContainerStarted","Data":"907ad46c32fe221593579c4deddeda768ae5282f3330068c382cb29fc7c93af2"}
Jul 21 04:10:31.451552 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.451525    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-99xh7" event={"ID":"e2b54ec0-f78b-4aa6-a64f-041017187539","Type":"ContainerStarted","Data":"0dc4ceabba327a5d78d71a8aafd84b447b74892675407755d1a89a4b68ced0c4"}
Jul 21 04:10:31.543250 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.543220    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:31.646679 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.646581    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:31.712824 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.712735    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:31.714963 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.714941    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"]
Jul 21 04:10:31.730945 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.730911    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"]
Jul 21 04:10:31.731081 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.731068    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"
Jul 21 04:10:31.734259 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.734237    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 21 04:10:31.734376 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.734282    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-l5c9k\""
Jul 21 04:10:31.875068 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.874968    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b968b540-a6d6-4a3c-a848-703d14667630-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pmzjh\" (UID: \"b968b540-a6d6-4a3c-a848-703d14667630\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"
Jul 21 04:10:31.956234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.956194    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:31.975406 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:31.975374    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b968b540-a6d6-4a3c-a848-703d14667630-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pmzjh\" (UID: \"b968b540-a6d6-4a3c-a848-703d14667630\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"
Jul 21 04:10:31.975594 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:31.975541    2546 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 21 04:10:31.975661 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:31.975606    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b968b540-a6d6-4a3c-a848-703d14667630-tls-certificates podName:b968b540-a6d6-4a3c-a848-703d14667630 nodeName:}" failed. No retries permitted until 2026-07-21 04:10:32.475581946 +0000 UTC m=+52.861070833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b968b540-a6d6-4a3c-a848-703d14667630-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-pmzjh" (UID: "b968b540-a6d6-4a3c-a848-703d14667630") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 21 04:10:32.056586 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:32.056557    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:32.479726 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:32.479688    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b968b540-a6d6-4a3c-a848-703d14667630-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pmzjh\" (UID: \"b968b540-a6d6-4a3c-a848-703d14667630\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"
Jul 21 04:10:32.480262 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:32.479864    2546 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 21 04:10:32.480262 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:32.479952    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b968b540-a6d6-4a3c-a848-703d14667630-tls-certificates podName:b968b540-a6d6-4a3c-a848-703d14667630 nodeName:}" failed. No retries permitted until 2026-07-21 04:10:33.479928778 +0000 UTC m=+53.865417682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b968b540-a6d6-4a3c-a848-703d14667630-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-pmzjh" (UID: "b968b540-a6d6-4a3c-a848-703d14667630") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 21 04:10:32.743839 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:32.743723    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:33.489016 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:33.488988    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b968b540-a6d6-4a3c-a848-703d14667630-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pmzjh\" (UID: \"b968b540-a6d6-4a3c-a848-703d14667630\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"
Jul 21 04:10:33.493103 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:33.493077    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b968b540-a6d6-4a3c-a848-703d14667630-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pmzjh\" (UID: \"b968b540-a6d6-4a3c-a848-703d14667630\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"
Jul 21 04:10:33.543480 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:33.543436    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"
Jul 21 04:10:35.681031 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:35.680988    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:35.745458 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:35.745429    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:36.654194 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:36.654107    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"]
Jul 21 04:10:36.655773 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:36.655742    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb968b540_a6d6_4a3c_a848_703d14667630.slice/crio-0bfecdb7b0e9d9c7080ede38d86149081461b89e0380a8dfb75060e3a22dbb4e WatchSource:0}: Error finding container 0bfecdb7b0e9d9c7080ede38d86149081461b89e0380a8dfb75060e3a22dbb4e: Status 404 returned error can't find the container with id 0bfecdb7b0e9d9c7080ede38d86149081461b89e0380a8dfb75060e3a22dbb4e
Jul 21 04:10:37.064455 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.064425    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:37.136293 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.136267    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:37.374876 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.374794    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-8qxcc"]
Jul 21 04:10:37.377162 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.377139    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-8qxcc"
Jul 21 04:10:37.379953 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.379929    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 04:10:37.380213 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.380194    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 04:10:37.380393 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.380376    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-lvggp\""
Jul 21 04:10:37.388594 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.388425    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-8qxcc"]
Jul 21 04:10:37.390778 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.390749    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:37.419014 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.418982    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqvx2\" (UniqueName: \"kubernetes.io/projected/bda4a0b7-0c94-4219-acbc-50ccead235f2-kube-api-access-rqvx2\") pod \"downloads-5bc67dcd46-8qxcc\" (UID: \"bda4a0b7-0c94-4219-acbc-50ccead235f2\") " pod="openshift-console/downloads-5bc67dcd46-8qxcc"
Jul 21 04:10:37.470800 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.470731    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh" event={"ID":"b968b540-a6d6-4a3c-a848-703d14667630","Type":"ContainerStarted","Data":"0bfecdb7b0e9d9c7080ede38d86149081461b89e0380a8dfb75060e3a22dbb4e"}
Jul 21 04:10:37.473887 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.473246    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-99xh7" event={"ID":"e2b54ec0-f78b-4aa6-a64f-041017187539","Type":"ContainerStarted","Data":"d6f2a9b67bc831e4bec29236e899216886101d361daa3441d069af1b9c6389b9"}
Jul 21 04:10:37.476775 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.476742    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qkv8k" event={"ID":"2915ad8b-22a5-489c-ba33-3f592a81241a","Type":"ContainerStarted","Data":"a88248a209b70f4fd2d69b19937d325916fa95d10af2c0a31b9337b4e4332456"}
Jul 21 04:10:37.476907 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.476793    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qkv8k" event={"ID":"2915ad8b-22a5-489c-ba33-3f592a81241a","Type":"ContainerStarted","Data":"ee0ece7cd5eb005f4ec187a84f35b5ef1d37dd53d87a8445419ac1a3074d8ebe"}
Jul 21 04:10:37.477319 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.477301    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:37.480126 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.480080    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnbk7" event={"ID":"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314","Type":"ContainerStarted","Data":"eba8a0d3b6e839829c2de4fb45e44ea65d85e94aed05dbe81bf2f932240bad26"}
Jul 21 04:10:37.482312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.482270    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:37.494394 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.494343    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-99xh7" podStartSLOduration=2.481770184 podStartE2EDuration="8.494326896s" totalImagesPullingTime="6.012556712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:37.49283637 +0000 UTC m=+57.878325283" watchObservedRunningTime="2026-07-21 04:10:37.494326896 +0000 UTC m=+57.879815806"
Jul 21 04:10:37.520870 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.519636    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqvx2\" (UniqueName: \"kubernetes.io/projected/bda4a0b7-0c94-4219-acbc-50ccead235f2-kube-api-access-rqvx2\") pod \"downloads-5bc67dcd46-8qxcc\" (UID: \"bda4a0b7-0c94-4219-acbc-50ccead235f2\") " pod="openshift-console/downloads-5bc67dcd46-8qxcc"
Jul 21 04:10:37.533027 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.532992    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqvx2\" (UniqueName: \"kubernetes.io/projected/bda4a0b7-0c94-4219-acbc-50ccead235f2-kube-api-access-rqvx2\") pod \"downloads-5bc67dcd46-8qxcc\" (UID: \"bda4a0b7-0c94-4219-acbc-50ccead235f2\") " pod="openshift-console/downloads-5bc67dcd46-8qxcc"
Jul 21 04:10:37.582554 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.582516    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:37.690568 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.690455    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:37.693197 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.692957    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-8qxcc"
Jul 21 04:10:37.986013 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.985952    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qkv8k" podStartSLOduration=2.048516342 podStartE2EDuration="7.985927848s" totalImagesPullingTime="5.937411506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:37.512399779 +0000 UTC m=+57.897888692" watchObservedRunningTime="2026-07-21 04:10:37.985927848 +0000 UTC m=+58.371416758"
Jul 21 04:10:37.987008 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:37.986977    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-8qxcc"]
Jul 21 04:10:37.990813 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:37.990785    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda4a0b7_0c94_4219_acbc_50ccead235f2.slice/crio-9d14b33db4a7f029ad8f660c1a22ed682aff16718537f3c030824a3fdf46d545 WatchSource:0}: Error finding container 9d14b33db4a7f029ad8f660c1a22ed682aff16718537f3c030824a3fdf46d545: Status 404 returned error can't find the container with id 9d14b33db4a7f029ad8f660c1a22ed682aff16718537f3c030824a3fdf46d545
Jul 21 04:10:38.080080 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.080045    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:38.463399 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.463366    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:38.488573 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.488531    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh" event={"ID":"b968b540-a6d6-4a3c-a848-703d14667630","Type":"ContainerStarted","Data":"6fc80478cebce80056932d9559c8257cf787db01a767369f58d77c6c6ecc36d2"}
Jul 21 04:10:38.488755 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.488736    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"
Jul 21 04:10:38.490201 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.490177    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-8qxcc" event={"ID":"bda4a0b7-0c94-4219-acbc-50ccead235f2","Type":"ContainerStarted","Data":"9d14b33db4a7f029ad8f660c1a22ed682aff16718537f3c030824a3fdf46d545"}
Jul 21 04:10:38.494294 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.494262    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh"
Jul 21 04:10:38.508958 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.508912    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pmzjh" podStartSLOduration=6.293208314 podStartE2EDuration="7.508898874s" totalImagesPullingTime="1.21569056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:38.508813845 +0000 UTC m=+58.894302939" watchObservedRunningTime="2026-07-21 04:10:38.508898874 +0000 UTC m=+58.894387782"
Jul 21 04:10:38.565525 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.565464    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:38.696791 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.696754    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:38.797558 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.797531    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-45jck"]
Jul 21 04:10:38.803885 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.803857    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:38.805381 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.805362    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.808102 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.808082    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 04:10:38.808215 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.808084    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 04:10:38.808450 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.808434    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 21 04:10:38.808450 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.808444    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 21 04:10:38.809284 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.809261    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 04:10:38.809436 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.809295    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7cp8m\""
Jul 21 04:10:38.813221 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.813093    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-45jck"]
Jul 21 04:10:38.829788 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.829769    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/081284ef-343b-4f90-9981-f23f723f5684-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.829891 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.829807    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpkz6\" (UniqueName: \"kubernetes.io/projected/081284ef-343b-4f90-9981-f23f723f5684-kube-api-access-fpkz6\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.829891 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.829836    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/081284ef-343b-4f90-9981-f23f723f5684-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.829891 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.829864    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/081284ef-343b-4f90-9981-f23f723f5684-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.931100 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.931065    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpkz6\" (UniqueName: \"kubernetes.io/projected/081284ef-343b-4f90-9981-f23f723f5684-kube-api-access-fpkz6\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.931308 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.931172    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/081284ef-343b-4f90-9981-f23f723f5684-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.931308 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.931224    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/081284ef-343b-4f90-9981-f23f723f5684-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.931389 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.931318    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/081284ef-343b-4f90-9981-f23f723f5684-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.932094 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.932074    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/081284ef-343b-4f90-9981-f23f723f5684-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.934027 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.933990    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/081284ef-343b-4f90-9981-f23f723f5684-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.934187 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.934153    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/081284ef-343b-4f90-9981-f23f723f5684-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:38.934285 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.934260    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:38.942018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:38.941994    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpkz6\" (UniqueName: \"kubernetes.io/projected/081284ef-343b-4f90-9981-f23f723f5684-kube-api-access-fpkz6\") pod \"prometheus-operator-5945b5cfd-45jck\" (UID: \"081284ef-343b-4f90-9981-f23f723f5684\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:39.064889 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:39.064820    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:39.116409 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:39.116376    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck"
Jul 21 04:10:39.284105 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:39.284052    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:39.482107 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:39.482071    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:40.472021 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:40.471986    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-45jck"]
Jul 21 04:10:40.475012 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:40.474970    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081284ef_343b_4f90_9981_f23f723f5684.slice/crio-e7541e11d95ad8a294575b9d2153aa17af16b66f0b7e19e38f63bfb74ebd8b40 WatchSource:0}: Error finding container e7541e11d95ad8a294575b9d2153aa17af16b66f0b7e19e38f63bfb74ebd8b40: Status 404 returned error can't find the container with id e7541e11d95ad8a294575b9d2153aa17af16b66f0b7e19e38f63bfb74ebd8b40
Jul 21 04:10:40.498260 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:40.498230    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck" event={"ID":"081284ef-343b-4f90-9981-f23f723f5684","Type":"ContainerStarted","Data":"e7541e11d95ad8a294575b9d2153aa17af16b66f0b7e19e38f63bfb74ebd8b40"}
Jul 21 04:10:40.500287 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:40.500220    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnbk7" event={"ID":"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314","Type":"ContainerStarted","Data":"c92d582e583166d21d2a68e487e32d90869b1766ab91a7248fc4f11ac1797ae9"}
Jul 21 04:10:40.679691 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:40.679617    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:40.766421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:40.766391    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:41.406664 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:41.406627    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp2zl"
Jul 21 04:10:41.540548 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:41.540514    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:41.606621 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:41.606590    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:42.272094 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:42.272011    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:42.509996 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:42.509419    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck" event={"ID":"081284ef-343b-4f90-9981-f23f723f5684","Type":"ContainerStarted","Data":"34957d3d894fb6a337d40c8a074bb5c277cb5fd818f4d69b918ade90df36fbd7"}
Jul 21 04:10:42.509996 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:42.509489    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck" event={"ID":"081284ef-343b-4f90-9981-f23f723f5684","Type":"ContainerStarted","Data":"b30f2724367661d271cb5ab8b41658a635f50b4c69c35eda3046a1ab36e6dd66"}
Jul 21 04:10:42.528471 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:42.528377    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-45jck" podStartSLOduration=3.023613352 podStartE2EDuration="4.528357931s" totalImagesPullingTime="1.504744579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:42.527701756 +0000 UTC m=+62.913190665" watchObservedRunningTime="2026-07-21 04:10:42.528357931 +0000 UTC m=+62.913846842"
Jul 21 04:10:42.951596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:42.951502    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:43.052027 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:43.051987    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:43.123166 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:43.123133    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:43.919570 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:43.919537    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:44.021585 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.021538    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:44.123034 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.123000    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:44.148780 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.148734    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"]
Jul 21 04:10:44.166476 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.166445    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"]
Jul 21 04:10:44.169769 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.168732    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.172040 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.172012    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 21 04:10:44.174022 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.172341    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 21 04:10:44.174022 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.172595    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-v9rf5\""
Jul 21 04:10:44.176514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.176491    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"]
Jul 21 04:10:44.182079 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.182059    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kl7wx"]
Jul 21 04:10:44.182270 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.182219    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.185046 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.185002    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 21 04:10:44.185275 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.185255    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 21 04:10:44.185275 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.185268    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-snr5x\""
Jul 21 04:10:44.185487 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.185470    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 21 04:10:44.189926 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.189894    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"]
Jul 21 04:10:44.190050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.190032    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.192639 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.192620    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zwp2w\""
Jul 21 04:10:44.194068 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.194046    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 04:10:44.194620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.194291    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 04:10:44.194620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.194473    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 04:10:44.204216 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.204190    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276363    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hv57\" (UniqueName: \"kubernetes.io/projected/878373ef-ec3e-44fe-8f96-57d91d72435e-kube-api-access-6hv57\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276415    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-sys\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276443    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276482    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms5zw\" (UniqueName: \"kubernetes.io/projected/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-kube-api-access-ms5zw\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276513    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276551    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-root\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276572    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-wtmp\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276602    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276629    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-textfile\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276653    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/966786ab-2d3c-49fa-b002-38d13d6eb388-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.276742 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276699    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/878373ef-ec3e-44fe-8f96-57d91d72435e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.277461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276761    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/966786ab-2d3c-49fa-b002-38d13d6eb388-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.277461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276810    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/878373ef-ec3e-44fe-8f96-57d91d72435e-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.277461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276832    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-metrics-client-ca\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.277461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276848    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wxjm\" (UniqueName: \"kubernetes.io/projected/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-api-access-6wxjm\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.277461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276905    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.277461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276942    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-accelerators-collector-config\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.277461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.276972    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-tls\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.277461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.277009    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/878373ef-ec3e-44fe-8f96-57d91d72435e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.283261 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.283238    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:44.378287 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378258    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-accelerators-collector-config\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378287 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378292    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-tls\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378314    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/878373ef-ec3e-44fe-8f96-57d91d72435e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.378519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378391    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hv57\" (UniqueName: \"kubernetes.io/projected/878373ef-ec3e-44fe-8f96-57d91d72435e-kube-api-access-6hv57\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.378519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378424    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-sys\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378444    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.378519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378468    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms5zw\" (UniqueName: \"kubernetes.io/projected/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-kube-api-access-ms5zw\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378499    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378525    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-root\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378546    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-sys\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378549    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-wtmp\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378611    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378645    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-textfile\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378665    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-wtmp\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378670    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/966786ab-2d3c-49fa-b002-38d13d6eb388-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378722    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/878373ef-ec3e-44fe-8f96-57d91d72435e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378748    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/966786ab-2d3c-49fa-b002-38d13d6eb388-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.378818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378803    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/878373ef-ec3e-44fe-8f96-57d91d72435e-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.379264 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378842    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-metrics-client-ca\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.379264 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378867    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wxjm\" (UniqueName: \"kubernetes.io/projected/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-api-access-6wxjm\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.379264 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378876    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-accelerators-collector-config\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.379264 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378940    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.379264 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.378942    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-root\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.379963 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.379939    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.380068 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.380046    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/878373ef-ec3e-44fe-8f96-57d91d72435e-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.380183 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:44.380165    2546 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 21 04:10:44.380245 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.380176    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-metrics-client-ca\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.380245 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:10:44.380240    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-tls podName:966786ab-2d3c-49fa-b002-38d13d6eb388 nodeName:}" failed. No retries permitted until 2026-07-21 04:10:44.880221377 +0000 UTC m=+65.265710266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-zd2lp" (UID: "966786ab-2d3c-49fa-b002-38d13d6eb388") : secret "kube-state-metrics-tls" not found
Jul 21 04:10:44.380352 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.380253    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/966786ab-2d3c-49fa-b002-38d13d6eb388-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.380352 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.380258    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-textfile\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.380474 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.380454    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/966786ab-2d3c-49fa-b002-38d13d6eb388-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.381410 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.381326    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.381552 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.381519    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/878373ef-ec3e-44fe-8f96-57d91d72435e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.382452 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.382414    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:44.382744 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.382724    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/878373ef-ec3e-44fe-8f96-57d91d72435e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.382809 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.382754    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-node-exporter-tls\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.385574 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.385545    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.391134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.391089    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms5zw\" (UniqueName: \"kubernetes.io/projected/63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf-kube-api-access-ms5zw\") pod \"node-exporter-kl7wx\" (UID: \"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf\") " pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.392275 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.392231    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wxjm\" (UniqueName: \"kubernetes.io/projected/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-api-access-6wxjm\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.392365 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.392303    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hv57\" (UniqueName: \"kubernetes.io/projected/878373ef-ec3e-44fe-8f96-57d91d72435e-kube-api-access-6hv57\") pod \"openshift-state-metrics-79f6958cdf-68cnf\" (UID: \"878373ef-ec3e-44fe-8f96-57d91d72435e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.482484 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.482450    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"
Jul 21 04:10:44.483669 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.483635    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:44.502699 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.502680    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kl7wx"
Jul 21 04:10:44.547037 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:44.546994    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b58ae4_5ef0_4f79_bb55_1cf8dc55f0cf.slice/crio-124bb5263b86142c7f90883a6c97f524aea3cac4469c67e90350ae673a92f322 WatchSource:0}: Error finding container 124bb5263b86142c7f90883a6c97f524aea3cac4469c67e90350ae673a92f322: Status 404 returned error can't find the container with id 124bb5263b86142c7f90883a6c97f524aea3cac4469c67e90350ae673a92f322
Jul 21 04:10:44.591063 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.591023    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:44.678964 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.678937    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf"]
Jul 21 04:10:44.682510 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:44.682484    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod878373ef_ec3e_44fe_8f96_57d91d72435e.slice/crio-34367ad23bcbef5b954a7879260f56066e0c2d2689163562a80386ccd66ad709 WatchSource:0}: Error finding container 34367ad23bcbef5b954a7879260f56066e0c2d2689163562a80386ccd66ad709: Status 404 returned error can't find the container with id 34367ad23bcbef5b954a7879260f56066e0c2d2689163562a80386ccd66ad709
Jul 21 04:10:44.683398 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.683378    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:44.883074 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.883035    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:44.885934 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:44.885912    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/966786ab-2d3c-49fa-b002-38d13d6eb388-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-zd2lp\" (UID: \"966786ab-2d3c-49fa-b002-38d13d6eb388\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:45.092072 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.091994    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"
Jul 21 04:10:45.156324 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.156237    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:45.255022 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.254994    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 04:10:45.257850 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.257823    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:45.260259 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.260236    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-zd2lp"]
Jul 21 04:10:45.260421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.260325    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.264272 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.264189    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 21 04:10:45.264272 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.264251    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 21 04:10:45.264272 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.264252    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 21 04:10:45.264500 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.264273    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 21 04:10:45.264500 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.264251    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 21 04:10:45.265058 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.264612    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 21 04:10:45.265058 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.264734    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x5czt\""
Jul 21 04:10:45.265058 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.264772    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 21 04:10:45.265058 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.264823    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 21 04:10:45.266229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.265256    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 21 04:10:45.277903 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.277824    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 04:10:45.287032 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287002    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287157 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287066    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-out\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287291 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287108    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287367 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287337    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86dgz\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-kube-api-access-86dgz\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287423 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287377    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287423 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287412    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287522 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287437    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-web-config\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287522 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287478    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287522 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287496    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287661 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287529    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287661 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287552    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287661 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287578    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.287661 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.287604    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.371411 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.371335    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:45.388354 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.388321    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.388513 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.388363    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.388513 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.388410    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.388513 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.388434    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.388513 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.388462    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.388513 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.388487    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.388513 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.388511    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.389171 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.389142    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.389311 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.389268    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.389512 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.389490    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-out\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.389611 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.389549    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.389611 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.389583    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86dgz\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-kube-api-access-86dgz\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.389717 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.389611    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.389717 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.389651    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.389717 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.389675    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-web-config\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.390821 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.390625    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.392386 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.392324    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.395165 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.395143    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.395855 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.395831    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.396105 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.396085    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-out\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.398421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.396268    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-web-config\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.398421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.396647    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.398421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.396997    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.398421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.397192    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.399064 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.399040    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.401931 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.401910    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86dgz\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-kube-api-access-86dgz\") pod \"alertmanager-main-0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.470131 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.470065    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:45.521992 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.521954    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnbk7" event={"ID":"e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314","Type":"ContainerStarted","Data":"7cba4a16a61bfe72dae20d7bcc8843a53a3401b551b3597a1917d7ce63b0968b"}
Jul 21 04:10:45.524416 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.524388    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf" event={"ID":"878373ef-ec3e-44fe-8f96-57d91d72435e","Type":"ContainerStarted","Data":"56c7455542b917626b2dfdcdc121d8e9a8d7820261a8815d1ef342ddcc29bf48"}
Jul 21 04:10:45.524569 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.524437    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf" event={"ID":"878373ef-ec3e-44fe-8f96-57d91d72435e","Type":"ContainerStarted","Data":"4cc233418dd0f95dc0cf85ca2359f7038a4fd02eecf1ff8386d17c68e02a496b"}
Jul 21 04:10:45.524569 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.524462    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf" event={"ID":"878373ef-ec3e-44fe-8f96-57d91d72435e","Type":"ContainerStarted","Data":"34367ad23bcbef5b954a7879260f56066e0c2d2689163562a80386ccd66ad709"}
Jul 21 04:10:45.525838 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.525814    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kl7wx" event={"ID":"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf","Type":"ContainerStarted","Data":"124bb5263b86142c7f90883a6c97f524aea3cac4469c67e90350ae673a92f322"}
Jul 21 04:10:45.544874 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.544808    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hnbk7" podStartSLOduration=2.632838463 podStartE2EDuration="16.544792033s" totalImagesPullingTime="13.91195357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:10:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:45.544671924 +0000 UTC m=+65.930160850" watchObservedRunningTime="2026-07-21 04:10:45.544792033 +0000 UTC m=+65.930280944"
Jul 21 04:10:45.568356 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:45.568328    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966786ab_2d3c_49fa_b002_38d13d6eb388.slice/crio-4af37475bd7484b3196759b3f19b67fbd4cbce4e443f1c5f50b1dee591903fba WatchSource:0}: Error finding container 4af37475bd7484b3196759b3f19b67fbd4cbce4e443f1c5f50b1dee591903fba: Status 404 returned error can't find the container with id 4af37475bd7484b3196759b3f19b67fbd4cbce4e443f1c5f50b1dee591903fba
Jul 21 04:10:45.574685 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.574663    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:10:45.748014 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.747985    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 04:10:45.752192 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:45.752160    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6cdb45_0c1b_4355_a698_321b7c2be6b0.slice/crio-31dff09f5dedea8501d18fc582e677db3ad392c586bdb6c73425dc0a0671cf24 WatchSource:0}: Error finding container 31dff09f5dedea8501d18fc582e677db3ad392c586bdb6c73425dc0a0671cf24: Status 404 returned error can't find the container with id 31dff09f5dedea8501d18fc582e677db3ad392c586bdb6c73425dc0a0671cf24
Jul 21 04:10:45.882693 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.882658    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79f8bb54cc-g2pv8"]
Jul 21 04:10:45.894680 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.894644    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:45.894912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.894728    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:45.896584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.896557    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79f8bb54cc-g2pv8"]
Jul 21 04:10:45.896741 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.896671    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:45.897559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.897534    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 04:10:45.897668 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.897534    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 04:10:45.899611 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.899553    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 04:10:45.899611 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.899600    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 04:10:45.899730 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.899621    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 04:10:45.899993 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.899975    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 04:10:45.900510 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.900265    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 04:10:45.900510 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.900360    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l6m6x\""
Jul 21 04:10:45.907796 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.907766    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f4d15b6-a854-4feb-82a8-3fc155affffb-original-pull-secret\") pod \"global-pull-secret-syncer-75ljs\" (UID: \"4f4d15b6-a854-4feb-82a8-3fc155affffb\") " pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:45.907902 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.907812    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15bb69f1-6e84-48be-92fb-fc48362465b7-metrics-certs\") pod \"network-metrics-daemon-c6kf7\" (UID: \"15bb69f1-6e84-48be-92fb-fc48362465b7\") " pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:45.995632 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.995510    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-config\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:45.995770 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.995742    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-service-ca\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:45.995886 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.995795    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-oauth-config\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:45.995886 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.995837    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:45.995886 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.995879    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-serving-cert\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:45.996042 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.995918    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-oauth-serving-cert\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:45.996042 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.995941    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpjw9\" (UniqueName: \"kubernetes.io/projected/b3c7d41a-71e2-4660-9ed1-b74b761886e5-kube-api-access-bpjw9\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:45.998326 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:45.998305    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 04:10:46.008617 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.008599    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 04:10:46.019521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.019498    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6px82\" (UniqueName: \"kubernetes.io/projected/05f84680-a4fc-4318-b064-6f28c832e6f5-kube-api-access-6px82\") pod \"network-check-target-xv2m6\" (UID: \"05f84680-a4fc-4318-b064-6f28c832e6f5\") " pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:46.097354 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.097236    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-oauth-serving-cert\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.097354 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.097276    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpjw9\" (UniqueName: \"kubernetes.io/projected/b3c7d41a-71e2-4660-9ed1-b74b761886e5-kube-api-access-bpjw9\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.097354 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.097323    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-config\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.097354 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.097354    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-service-ca\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.097806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.097381    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-oauth-config\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.097806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.097435    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-serving-cert\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.098528 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.098499    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-config\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.098619 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.098499    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-oauth-serving-cert\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.098619 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.098510    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-service-ca\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.100037 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.100008    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-serving-cert\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.100633 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.100607    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-oauth-config\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.106732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.106709    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpjw9\" (UniqueName: \"kubernetes.io/projected/b3c7d41a-71e2-4660-9ed1-b74b761886e5-kube-api-access-bpjw9\") pod \"console-79f8bb54cc-g2pv8\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") " pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.153963 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.153870    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"]
Jul 21 04:10:46.174229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.174200    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"]
Jul 21 04:10:46.174491 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.174473    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.174621 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.174600    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9v8hm\""
Jul 21 04:10:46.177466 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.177444    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 21 04:10:46.179055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.177743    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6dd64d4l2ssre\""
Jul 21 04:10:46.179055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.177985    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 21 04:10:46.179055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.178197    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 21 04:10:46.179055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.178373    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 21 04:10:46.179055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.178648    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q2s2m\""
Jul 21 04:10:46.179055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.178879    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 21 04:10:46.182511 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.182319    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6kf7"
Jul 21 04:10:46.182836 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.182814    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qwqtb\""
Jul 21 04:10:46.187873 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.187858    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75ljs"
Jul 21 04:10:46.191435 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.191414    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:10:46.208650 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.208589    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:10:46.301864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.301369    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.301864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.301420    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1f82002-4d5f-41b0-89dd-7673c1df3111-metrics-client-ca\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.301864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.301492    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-grpc-tls\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.301864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.301542    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.301864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.301584    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.301864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.301614    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-tls\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.301864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.301648    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrn8c\" (UniqueName: \"kubernetes.io/projected/b1f82002-4d5f-41b0-89dd-7673c1df3111-kube-api-access-hrn8c\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.301864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.301709    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.386429 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.386326    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c6kf7"]
Jul 21 04:10:46.403380 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.403342    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-grpc-tls\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.407783 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.407755    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-75ljs"]
Jul 21 04:10:46.409266 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.409221    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.409349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.409301    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.409402 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.409349    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-tls\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.409402 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.409380    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrn8c\" (UniqueName: \"kubernetes.io/projected/b1f82002-4d5f-41b0-89dd-7673c1df3111-kube-api-access-hrn8c\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.409508 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.409454    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.409508 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.409501    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.409603 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.409525    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1f82002-4d5f-41b0-89dd-7673c1df3111-metrics-client-ca\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.410528 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.410475    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1f82002-4d5f-41b0-89dd-7673c1df3111-metrics-client-ca\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.412521 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:46.412478    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4d15b6_a854_4feb_82a8_3fc155affffb.slice/crio-b157c3a375c78d0b1a586ea5de64775fccad25bbe7b342e2f4bdfa462e662f6f WatchSource:0}: Error finding container b157c3a375c78d0b1a586ea5de64775fccad25bbe7b342e2f4bdfa462e662f6f: Status 404 returned error can't find the container with id b157c3a375c78d0b1a586ea5de64775fccad25bbe7b342e2f4bdfa462e662f6f
Jul 21 04:10:46.427468 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.421016    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.446973 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.441214    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xv2m6"]
Jul 21 04:10:46.446973 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.442190    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.448688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.447506    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.448688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.447105    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-tls\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.449368 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.449317    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.450006 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.449959    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b1f82002-4d5f-41b0-89dd-7673c1df3111-secret-grpc-tls\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.452493 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.452451    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrn8c\" (UniqueName: \"kubernetes.io/projected/b1f82002-4d5f-41b0-89dd-7673c1df3111-kube-api-access-hrn8c\") pod \"thanos-querier-5884f7ddfb-twz9t\" (UID: \"b1f82002-4d5f-41b0-89dd-7673c1df3111\") " pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.456480 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:46.456446    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f84680_a4fc_4318_b064_6f28c832e6f5.slice/crio-41173a9fecdc87f96a29bdefb36f0553b7d0dad0e0ee9127d106b7903f9fa692 WatchSource:0}: Error finding container 41173a9fecdc87f96a29bdefb36f0553b7d0dad0e0ee9127d106b7903f9fa692: Status 404 returned error can't find the container with id 41173a9fecdc87f96a29bdefb36f0553b7d0dad0e0ee9127d106b7903f9fa692
Jul 21 04:10:46.492495 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.489535    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:10:46.492495 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:46.489607    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c7d41a_71e2_4660_9ed1_b74b761886e5.slice/crio-5b31cf4cc031c32ae3a2581835747081b312334deb6abdaf8d33766a75c91cca WatchSource:0}: Error finding container 5b31cf4cc031c32ae3a2581835747081b312334deb6abdaf8d33766a75c91cca: Status 404 returned error can't find the container with id 5b31cf4cc031c32ae3a2581835747081b312334deb6abdaf8d33766a75c91cca
Jul 21 04:10:46.492495 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.491923    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79f8bb54cc-g2pv8"]
Jul 21 04:10:46.531198 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.531157    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp" event={"ID":"966786ab-2d3c-49fa-b002-38d13d6eb388","Type":"ContainerStarted","Data":"4af37475bd7484b3196759b3f19b67fbd4cbce4e443f1c5f50b1dee591903fba"}
Jul 21 04:10:46.532995 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.532940    2546 generic.go:356] "Generic (PLEG): container finished" podID="63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf" containerID="a4a87358517dafb22ee352be35954244d1cbc11225d04ea55eb68a3a4893fb2a" exitCode=0
Jul 21 04:10:46.533091 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.533043    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kl7wx" event={"ID":"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf","Type":"ContainerDied","Data":"a4a87358517dafb22ee352be35954244d1cbc11225d04ea55eb68a3a4893fb2a"}
Jul 21 04:10:46.535793 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.535762    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xv2m6" event={"ID":"05f84680-a4fc-4318-b064-6f28c832e6f5","Type":"ContainerStarted","Data":"41173a9fecdc87f96a29bdefb36f0553b7d0dad0e0ee9127d106b7903f9fa692"}
Jul 21 04:10:46.537597 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.537567    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-75ljs" event={"ID":"4f4d15b6-a854-4feb-82a8-3fc155affffb","Type":"ContainerStarted","Data":"b157c3a375c78d0b1a586ea5de64775fccad25bbe7b342e2f4bdfa462e662f6f"}
Jul 21 04:10:46.539281 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.539254    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerStarted","Data":"31dff09f5dedea8501d18fc582e677db3ad392c586bdb6c73425dc0a0671cf24"}
Jul 21 04:10:46.540912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.540884    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79f8bb54cc-g2pv8" event={"ID":"b3c7d41a-71e2-4660-9ed1-b74b761886e5","Type":"ContainerStarted","Data":"5b31cf4cc031c32ae3a2581835747081b312334deb6abdaf8d33766a75c91cca"}
Jul 21 04:10:46.542522 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.542461    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c6kf7" event={"ID":"15bb69f1-6e84-48be-92fb-fc48362465b7","Type":"ContainerStarted","Data":"35663e135933337dd8b229c10cda50d64213a60f9f9b1c9b4e3d21c299d79ffc"}
Jul 21 04:10:46.697761 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:46.697685    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"]
Jul 21 04:10:47.100620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:47.100585    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:47.200447 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:47.200415    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:47.303548 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:47.303512    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:47.403694 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:47.403623    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:47.459652 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:47.459613    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f82002_4d5f_41b0_89dd_7673c1df3111.slice/crio-b133536d97257f8089e5f242aff7c4291270c049a3655022d3e8195ce801a688 WatchSource:0}: Error finding container b133536d97257f8089e5f242aff7c4291270c049a3655022d3e8195ce801a688: Status 404 returned error can't find the container with id b133536d97257f8089e5f242aff7c4291270c049a3655022d3e8195ce801a688
Jul 21 04:10:47.552700 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:47.552584    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t" event={"ID":"b1f82002-4d5f-41b0-89dd-7673c1df3111","Type":"ContainerStarted","Data":"b133536d97257f8089e5f242aff7c4291270c049a3655022d3e8195ce801a688"}
Jul 21 04:10:48.496629 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.495856    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qkv8k"
Jul 21 04:10:48.500727 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.499384    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7f44576f59-v5zpb"]
Jul 21 04:10:48.502555 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.502064    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.506384 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.505797    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-cjv6j\""
Jul 21 04:10:48.506384 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.505872    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 21 04:10:48.506384 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.506017    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bi6ostlj9guq7\""
Jul 21 04:10:48.506384 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.506234    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 21 04:10:48.506633 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.506584    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 21 04:10:48.506869 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.506845    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 21 04:10:48.513208 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.513183    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f44576f59-v5zpb"]
Jul 21 04:10:48.633084 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.631993    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0e709af-3e64-43c7-a199-552fac76dc2d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.633084 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.632098    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f0e709af-3e64-43c7-a199-552fac76dc2d-secret-metrics-server-tls\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.633084 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.632153    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f0e709af-3e64-43c7-a199-552fac76dc2d-audit-log\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.633084 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.632212    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw6xs\" (UniqueName: \"kubernetes.io/projected/f0e709af-3e64-43c7-a199-552fac76dc2d-kube-api-access-xw6xs\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.633084 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.632266    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f0e709af-3e64-43c7-a199-552fac76dc2d-client-ca-bundle\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.633084 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.632291    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f0e709af-3e64-43c7-a199-552fac76dc2d-secret-metrics-server-client-certs\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.633084 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.632320    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f0e709af-3e64-43c7-a199-552fac76dc2d-metrics-server-audit-profiles\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.736050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.733255    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f0e709af-3e64-43c7-a199-552fac76dc2d-secret-metrics-server-tls\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.736050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.734258    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f0e709af-3e64-43c7-a199-552fac76dc2d-audit-log\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.736050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.733744    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f0e709af-3e64-43c7-a199-552fac76dc2d-audit-log\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.736050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.734427    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw6xs\" (UniqueName: \"kubernetes.io/projected/f0e709af-3e64-43c7-a199-552fac76dc2d-kube-api-access-xw6xs\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.736050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.734809    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f0e709af-3e64-43c7-a199-552fac76dc2d-client-ca-bundle\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.736050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.734911    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f0e709af-3e64-43c7-a199-552fac76dc2d-secret-metrics-server-client-certs\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.736050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.735872    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f0e709af-3e64-43c7-a199-552fac76dc2d-metrics-server-audit-profiles\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.737833 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.737161    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0e709af-3e64-43c7-a199-552fac76dc2d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.737833 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.736906    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f0e709af-3e64-43c7-a199-552fac76dc2d-metrics-server-audit-profiles\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.737999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.737940    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f0e709af-3e64-43c7-a199-552fac76dc2d-secret-metrics-server-client-certs\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.738231 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.738209    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0e709af-3e64-43c7-a199-552fac76dc2d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.744191 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.743886    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f0e709af-3e64-43c7-a199-552fac76dc2d-client-ca-bundle\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.750713 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.750615    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f0e709af-3e64-43c7-a199-552fac76dc2d-secret-metrics-server-tls\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.763075 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.763036    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw6xs\" (UniqueName: \"kubernetes.io/projected/f0e709af-3e64-43c7-a199-552fac76dc2d-kube-api-access-xw6xs\") pod \"metrics-server-7f44576f59-v5zpb\" (UID: \"f0e709af-3e64-43c7-a199-552fac76dc2d\") " pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.850840 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.850812    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:10:48.937867 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.937796    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:48.973828 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.971403    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"]
Jul 21 04:10:48.982886 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.981353    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:10:48.985472 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.985406    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"]
Jul 21 04:10:48.988291 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.988097    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lkphw\""
Jul 21 04:10:48.988928 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:48.988277    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 21 04:10:49.017425 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.017291    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:49.041613 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.041574    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7a345e9-5838-4a63-9905-027db812c6af-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pfpwp\" (UID: \"c7a345e9-5838-4a63-9905-027db812c6af\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:10:49.041774 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.041729    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgkwk\" (UniqueName: \"kubernetes.io/projected/c7a345e9-5838-4a63-9905-027db812c6af-kube-api-access-dgkwk\") pod \"monitoring-plugin-7b7b7bc8bf-pfpwp\" (UID: \"c7a345e9-5838-4a63-9905-027db812c6af\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:10:49.081160 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.081108    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f44576f59-v5zpb"]
Jul 21 04:10:49.084655 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.084508    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:49.086188 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:49.086145    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e709af_3e64_43c7_a199_552fac76dc2d.slice/crio-3c3ccf5b95fa318e43618a397a5533605eeb979a8ff6ce51c94720926fff0a48 WatchSource:0}: Error finding container 3c3ccf5b95fa318e43618a397a5533605eeb979a8ff6ce51c94720926fff0a48: Status 404 returned error can't find the container with id 3c3ccf5b95fa318e43618a397a5533605eeb979a8ff6ce51c94720926fff0a48
Jul 21 04:10:49.143344 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.142402    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7a345e9-5838-4a63-9905-027db812c6af-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pfpwp\" (UID: \"c7a345e9-5838-4a63-9905-027db812c6af\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:10:49.143344 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.142517    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgkwk\" (UniqueName: \"kubernetes.io/projected/c7a345e9-5838-4a63-9905-027db812c6af-kube-api-access-dgkwk\") pod \"monitoring-plugin-7b7b7bc8bf-pfpwp\" (UID: \"c7a345e9-5838-4a63-9905-027db812c6af\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:10:49.147463 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.147411    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7a345e9-5838-4a63-9905-027db812c6af-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pfpwp\" (UID: \"c7a345e9-5838-4a63-9905-027db812c6af\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:10:49.151511 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.151451    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgkwk\" (UniqueName: \"kubernetes.io/projected/c7a345e9-5838-4a63-9905-027db812c6af-kube-api-access-dgkwk\") pod \"monitoring-plugin-7b7b7bc8bf-pfpwp\" (UID: \"c7a345e9-5838-4a63-9905-027db812c6af\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:10:49.171508 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.171444    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:49.301749 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.301295    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:10:49.378180 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.376385    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8b5f7967d-nsljl"]
Jul 21 04:10:49.380871 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.380191    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.385144 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.382827    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 21 04:10:49.385144 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.383382    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 21 04:10:49.385144 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.383558    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 21 04:10:49.385144 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.383756    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rs6lv\""
Jul 21 04:10:49.385144 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.383978    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 21 04:10:49.385144 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.384690    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 21 04:10:49.393591 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.393351    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8b5f7967d-nsljl"]
Jul 21 04:10:49.406323 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.406280    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 21 04:10:49.444785 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.444743    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-federate-client-tls\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.444926 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.444802    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.444926 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.444841    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/792fd228-fa50-4842-81ed-727eb3f9639d-serving-certs-ca-bundle\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.444926 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.444867    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-telemeter-client-tls\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.444926 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.444900    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-secret-telemeter-client\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.445174 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.444932    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/792fd228-fa50-4842-81ed-727eb3f9639d-metrics-client-ca\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.445174 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.444958    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frr7q\" (UniqueName: \"kubernetes.io/projected/792fd228-fa50-4842-81ed-727eb3f9639d-kube-api-access-frr7q\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.445174 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.445024    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/792fd228-fa50-4842-81ed-727eb3f9639d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.507701 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.507667    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"]
Jul 21 04:10:49.522777 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:49.522742    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a345e9_5838_4a63_9905_027db812c6af.slice/crio-6725e5b92ffef88f59d722481e52b8441ceffdf69bacd35a7560486965b28374 WatchSource:0}: Error finding container 6725e5b92ffef88f59d722481e52b8441ceffdf69bacd35a7560486965b28374: Status 404 returned error can't find the container with id 6725e5b92ffef88f59d722481e52b8441ceffdf69bacd35a7560486965b28374
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.546240    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-federate-client-tls\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.546288    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.546325    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/792fd228-fa50-4842-81ed-727eb3f9639d-serving-certs-ca-bundle\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.546353    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-telemeter-client-tls\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.546384    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-secret-telemeter-client\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.546420    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/792fd228-fa50-4842-81ed-727eb3f9639d-metrics-client-ca\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.546444    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frr7q\" (UniqueName: \"kubernetes.io/projected/792fd228-fa50-4842-81ed-727eb3f9639d-kube-api-access-frr7q\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.546511    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/792fd228-fa50-4842-81ed-727eb3f9639d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.547714    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/792fd228-fa50-4842-81ed-727eb3f9639d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.548477    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/792fd228-fa50-4842-81ed-727eb3f9639d-metrics-client-ca\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.550822 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.550748    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/792fd228-fa50-4842-81ed-727eb3f9639d-serving-certs-ca-bundle\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.552344 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.552233    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-secret-telemeter-client\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.554004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.553938    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-telemeter-client-tls\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.554102 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.554023    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.554640 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.554600    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/792fd228-fa50-4842-81ed-727eb3f9639d-federate-client-tls\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.557745 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.557572    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frr7q\" (UniqueName: \"kubernetes.io/projected/792fd228-fa50-4842-81ed-727eb3f9639d-kube-api-access-frr7q\") pod \"telemeter-client-8b5f7967d-nsljl\" (UID: \"792fd228-fa50-4842-81ed-727eb3f9639d\") " pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.573324 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.572409    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kl7wx" event={"ID":"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf","Type":"ContainerStarted","Data":"9340fb70c1481a10ddaa0bf625b51f89beeb0f93c0f296ae3d49dca56f9852a9"}
Jul 21 04:10:49.573324 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.572473    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kl7wx" event={"ID":"63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf","Type":"ContainerStarted","Data":"961756c7b1935ae6d0d6c2a05473dffeea3a832d508e25ad5ea32ae9f1225b63"}
Jul 21 04:10:49.578599 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.578515    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerDied","Data":"3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723"}
Jul 21 04:10:49.579133 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.578876    2546 generic.go:356] "Generic (PLEG): container finished" podID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerID="3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723" exitCode=0
Jul 21 04:10:49.584561 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.584519    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c6kf7" event={"ID":"15bb69f1-6e84-48be-92fb-fc48362465b7","Type":"ContainerStarted","Data":"c052b33eacb4422fd5194b67296bd855f3eebb283cb31257597b91c1e3910ddb"}
Jul 21 04:10:49.584646 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.584575    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c6kf7" event={"ID":"15bb69f1-6e84-48be-92fb-fc48362465b7","Type":"ContainerStarted","Data":"93e061646eb7c0d4a21cc410d4a00b6211273410c48d3f80618c96328c0e4709"}
Jul 21 04:10:49.590530 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.590503    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf" event={"ID":"878373ef-ec3e-44fe-8f96-57d91d72435e","Type":"ContainerStarted","Data":"4608a1d1d50fa7556beb505e01c48b8f659947216ee31a20bcecc94608e893f2"}
Jul 21 04:10:49.592815 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.592768    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kl7wx" podStartSLOduration=4.082127902 podStartE2EDuration="5.592752024s" totalImagesPullingTime="1.384720445s" totalInitContainerRuntime="125.903677ms" isStatefulPod=true podCreationTimestamp="2026-07-21 04:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:49.591593524 +0000 UTC m=+69.977082434" watchObservedRunningTime="2026-07-21 04:10:49.592752024 +0000 UTC m=+69.978240929"
Jul 21 04:10:49.597685 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.597656    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp" event={"ID":"966786ab-2d3c-49fa-b002-38d13d6eb388","Type":"ContainerStarted","Data":"95ea882ab2bf06416cf011f1e558fd1a15f656bb32725df34d7c74f11d410bed"}
Jul 21 04:10:49.597775 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.597711    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp" event={"ID":"966786ab-2d3c-49fa-b002-38d13d6eb388","Type":"ContainerStarted","Data":"2177fb661c06835854a376c4b5945c7463f6acd0cf9dac5730cfe2037db4ba1d"}
Jul 21 04:10:49.597775 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.597739    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp" event={"ID":"966786ab-2d3c-49fa-b002-38d13d6eb388","Type":"ContainerStarted","Data":"f8a6580716a6d7d3a840952aa2532bcff9e4248f21baf594e7a127e2fc8a6dfa"}
Jul 21 04:10:49.603239 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.603169    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp" event={"ID":"c7a345e9-5838-4a63-9905-027db812c6af","Type":"ContainerStarted","Data":"6725e5b92ffef88f59d722481e52b8441ceffdf69bacd35a7560486965b28374"}
Jul 21 04:10:49.606061 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.606016    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb" event={"ID":"f0e709af-3e64-43c7-a199-552fac76dc2d","Type":"ContainerStarted","Data":"3c3ccf5b95fa318e43618a397a5533605eeb979a8ff6ce51c94720926fff0a48"}
Jul 21 04:10:49.609559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.608214    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c6kf7" podStartSLOduration=67.496509835 podStartE2EDuration="1m9.608197497s" totalImagesPullingTime="2.111687662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:49.607838194 +0000 UTC m=+69.993327115" watchObservedRunningTime="2026-07-21 04:10:49.608197497 +0000 UTC m=+69.993686408"
Jul 21 04:10:49.668708 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.668428    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-68cnf" podStartSLOduration=2.010666033 podStartE2EDuration="5.668405574s" totalImagesPullingTime="3.657739541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:49.666388463 +0000 UTC m=+70.051877388" watchObservedRunningTime="2026-07-21 04:10:49.668405574 +0000 UTC m=+70.053894466"
Jul 21 04:10:49.703563 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.703531    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl"
Jul 21 04:10:49.713803 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.713773    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:49.771290 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.771255    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:49.906417 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.905169    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-zd2lp" podStartSLOduration=3.011901651 podStartE2EDuration="5.905145126s" totalImagesPullingTime="2.893243475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:10:49.689367258 +0000 UTC m=+70.074856168" watchObservedRunningTime="2026-07-21 04:10:49.905145126 +0000 UTC m=+70.290634037"
Jul 21 04:10:49.906417 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:49.905571    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8b5f7967d-nsljl"]
Jul 21 04:10:49.914283 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:10:49.914205    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792fd228_fa50_4842_81ed_727eb3f9639d.slice/crio-e11d6f88ddb6f9800ccd62fcf2bcffc3c7cb31c0e72578fa5859d1b1a62970d8 WatchSource:0}: Error finding container e11d6f88ddb6f9800ccd62fcf2bcffc3c7cb31c0e72578fa5859d1b1a62970d8: Status 404 returned error can't find the container with id e11d6f88ddb6f9800ccd62fcf2bcffc3c7cb31c0e72578fa5859d1b1a62970d8
Jul 21 04:10:50.469620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.469533    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:50.470314 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.470288    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 04:10:50.479152 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.477996    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.484567 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.483743    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 21 04:10:50.484567 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.483798    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 21 04:10:50.484567 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.483752    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5060cv17crf5t\""
Jul 21 04:10:50.484567 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.484029    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 21 04:10:50.484567 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.484223    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 21 04:10:50.484567 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.484385    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 21 04:10:50.486343 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.485001    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 21 04:10:50.486343 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.485155    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2pkdp\""
Jul 21 04:10:50.486343 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.485387    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 21 04:10:50.486343 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.485617    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 21 04:10:50.486343 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.485851    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 21 04:10:50.486343 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.485157    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 21 04:10:50.486343 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.486241    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 21 04:10:50.486734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.486548    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 21 04:10:50.492669 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.489952    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 21 04:10:50.493341 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.493250    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 04:10:50.494134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.494099    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.562708    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.562760    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.562795    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.562836    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.562863    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.562890    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-config-out\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.562920    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.562950    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.562980    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563011    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-web-config\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563040    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563076    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-config\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563099    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563143    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563181    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.563460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563218    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.564666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563244    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.564666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563273    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.564666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563315    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9zjv\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-kube-api-access-r9zjv\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.564666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.563406    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.635950 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.635822    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl" event={"ID":"792fd228-fa50-4842-81ed-727eb3f9639d","Type":"ContainerStarted","Data":"e11d6f88ddb6f9800ccd62fcf2bcffc3c7cb31c0e72578fa5859d1b1a62970d8"}
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.664790    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.664844    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.664873    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.664898    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-web-config\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.664928    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.664963    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-config\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.664985    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665010    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665033    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665060    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665086    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665134    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665163    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9zjv\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-kube-api-access-r9zjv\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665197    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665253    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665288    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.665688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.665316    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.669928 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.666276    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.669928 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.668400    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.669928 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.668716    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.669928 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.668752    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.669928 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.668782    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-config-out\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.669928 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.669086    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.669928 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.669720    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.670286 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.670173    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.675285 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.674958    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.677241 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.676946    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.687505 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.687416    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.693379 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.693236    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9zjv\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-kube-api-access-r9zjv\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.693771 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.693714    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.693771 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.693729    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.694894 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.694458    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.694894 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.694847    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-config\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.695263 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.695224    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-config-out\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.695515 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.695440    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.695783 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.695742    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-web-config\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.696270 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.696041    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.696536 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.696490    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.697182 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.697063    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.698688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.698666    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.820474 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.820375    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:10:50.916107 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:50.916066    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:51.017007 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:51.016937    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:52.275504 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:52.275467    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:52.339835 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:52.339796    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:54.178836 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:54.178790    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:54.247699 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:54.247670    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:54.630640 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:54.630601    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:54.691534 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:54.691504    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:55.496863 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:55.496828    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:55.598258 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:55.598196    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:57.085622 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:57.085585    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:57.186897 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:57.186863    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:59.469853 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:59.469814    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:59.580397 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:59.580359    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:10:59.966414 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:10:59.966329    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:00.067522 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:00.067483    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:03.703688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:03.703549    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:03.804400 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:03.804368    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:04.960452 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:04.960412    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79f8bb54cc-g2pv8"]
Jul 21 04:11:05.071044 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:05.071010    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:05.172941 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:05.172909    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:05.271328 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:05.271300    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:05.474404 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:05.474380    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 04:11:05.549823 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:11:05.549791    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46eeae26_1bda_4a50_8430_1cd5487760b0.slice/crio-827816675f02d2e1b5c7fbc17e63a0afeb007b0ca9dbfcfa30c43c42dd94f28a WatchSource:0}: Error finding container 827816675f02d2e1b5c7fbc17e63a0afeb007b0ca9dbfcfa30c43c42dd94f28a: Status 404 returned error can't find the container with id 827816675f02d2e1b5c7fbc17e63a0afeb007b0ca9dbfcfa30c43c42dd94f28a
Jul 21 04:11:05.701729 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:05.701554    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerStarted","Data":"827816675f02d2e1b5c7fbc17e63a0afeb007b0ca9dbfcfa30c43c42dd94f28a"}
Jul 21 04:11:05.728229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:05.727823    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79f8bb54cc-g2pv8" event={"ID":"b3c7d41a-71e2-4660-9ed1-b74b761886e5","Type":"ContainerStarted","Data":"2865e948bca19532f6263a218b4277ed9a6c754585ef3453c54389c9b5a55918"}
Jul 21 04:11:05.736074 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:05.735950    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t" event={"ID":"b1f82002-4d5f-41b0-89dd-7673c1df3111","Type":"ContainerStarted","Data":"b2925a6afdfdd606371ba817cd81cf2b1de3b247df15d38799210561c5389046"}
Jul 21 04:11:05.909975 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:05.909818    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:06.009955 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.009923    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:06.746291 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.745398    2546 generic.go:356] "Generic (PLEG): container finished" podID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerID="cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37" exitCode=0
Jul 21 04:11:06.746291 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.745509    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerDied","Data":"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"}
Jul 21 04:11:06.750386 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.749864    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xv2m6" event={"ID":"05f84680-a4fc-4318-b064-6f28c832e6f5","Type":"ContainerStarted","Data":"2148598d6384f9bf5f48b2a6f5af825318ef82aca13ad0b41e9afb2de004bffb"}
Jul 21 04:11:06.750386 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.750310    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:11:06.757770 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.757735    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerStarted","Data":"935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef"}
Jul 21 04:11:06.758108 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.757806    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerStarted","Data":"b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6"}
Jul 21 04:11:06.758108 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.757847    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerStarted","Data":"1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c"}
Jul 21 04:11:06.758108 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.757886    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerStarted","Data":"a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e"}
Jul 21 04:11:06.758108 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.757926    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerStarted","Data":"f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b"}
Jul 21 04:11:06.760731 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.760700    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp" event={"ID":"c7a345e9-5838-4a63-9905-027db812c6af","Type":"ContainerStarted","Data":"c80688b04a1d91aa0653db9ecc2afc1646f9fe13208aa521f625e2695aacaadf"}
Jul 21 04:11:06.761399 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.761344    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:11:06.767831 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.767371    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb" event={"ID":"f0e709af-3e64-43c7-a199-552fac76dc2d","Type":"ContainerStarted","Data":"41aa0b69866763afc9dcca2f95d96baf0f731170f7314dc7f64889d99bbc2f11"}
Jul 21 04:11:06.769223 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.769203    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp"
Jul 21 04:11:06.770722 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.770699    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-8qxcc" event={"ID":"bda4a0b7-0c94-4219-acbc-50ccead235f2","Type":"ContainerStarted","Data":"099506ddf5c89a43f588fa58c6852ce0c0c9e12a027070ba25f0a61bb116e258"}
Jul 21 04:11:06.773102 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.773075    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-8qxcc"
Jul 21 04:11:06.775795 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.775766    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t" event={"ID":"b1f82002-4d5f-41b0-89dd-7673c1df3111","Type":"ContainerStarted","Data":"bd0ddc670fdc784ac38de467703683c432f4ab9eb0f62121594f4075a52b5622"}
Jul 21 04:11:06.775899 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.775837    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t" event={"ID":"b1f82002-4d5f-41b0-89dd-7673c1df3111","Type":"ContainerStarted","Data":"d235ebe4e33403d8e0678ea0d41dd29938e65c48399b8395b52727f23322150e"}
Jul 21 04:11:06.779214 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.779186    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-75ljs" event={"ID":"4f4d15b6-a854-4feb-82a8-3fc155affffb","Type":"ContainerStarted","Data":"f605959a8037015ff49856c19f418277d6e8837e10e7bfeed07b67727b2ee934"}
Jul 21 04:11:06.783631 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.783580    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl" event={"ID":"792fd228-fa50-4842-81ed-727eb3f9639d","Type":"ContainerStarted","Data":"59d4c18cc0546f5b7a6b6e6bb09d33e180b616ae8043c77b99c3cd0d5f7e5ae8"}
Jul 21 04:11:06.783780 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.783634    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl" event={"ID":"792fd228-fa50-4842-81ed-727eb3f9639d","Type":"ContainerStarted","Data":"939137772bb375bc1a3d1e063bc2174afdc13187dde1328cb64562eff62cd743"}
Jul 21 04:11:06.783780 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.783663    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl" event={"ID":"792fd228-fa50-4842-81ed-727eb3f9639d","Type":"ContainerStarted","Data":"b363c2093ae504944cc19a4b8cee2b0e9705e445f25ee72e9cb27b64c23775a9"}
Jul 21 04:11:06.789010 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.788989    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-8qxcc"
Jul 21 04:11:06.816759 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.816699    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xv2m6" podStartSLOduration=67.939260343 podStartE2EDuration="1m26.81667404s" totalImagesPullingTime="18.877413697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:06.796263304 +0000 UTC m=+87.181752213" watchObservedRunningTime="2026-07-21 04:11:06.81667404 +0000 UTC m=+87.202162949"
Jul 21 04:11:06.841553 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.841431    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb" podStartSLOduration=2.597179725 podStartE2EDuration="18.841401741s" totalImagesPullingTime="16.244222016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:06.817836425 +0000 UTC m=+87.203325337" watchObservedRunningTime="2026-07-21 04:11:06.841401741 +0000 UTC m=+87.226890632"
Jul 21 04:11:06.861503 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.861448    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-8qxcc" podStartSLOduration=2.208004622 podStartE2EDuration="29.86142338s" totalImagesPullingTime="27.653418758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:06.841219373 +0000 UTC m=+87.226708294" watchObservedRunningTime="2026-07-21 04:11:06.86142338 +0000 UTC m=+87.246912290"
Jul 21 04:11:06.895062 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.893560    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8b5f7967d-nsljl" podStartSLOduration=2.18807262 podStartE2EDuration="17.893545081s" totalImagesPullingTime="15.705472461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:06.891284594 +0000 UTC m=+87.276773504" watchObservedRunningTime="2026-07-21 04:11:06.893545081 +0000 UTC m=+87.279033991"
Jul 21 04:11:06.895062 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.894086    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pfpwp" podStartSLOduration=3.080948194 podStartE2EDuration="18.894071495s" totalImagesPullingTime="15.813123301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:06.862620615 +0000 UTC m=+87.248109526" watchObservedRunningTime="2026-07-21 04:11:06.894071495 +0000 UTC m=+87.279560406"
Jul 21 04:11:06.915203 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.914447    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79f8bb54cc-g2pv8" podStartSLOduration=3.070401509 podStartE2EDuration="21.914426108s" totalImagesPullingTime="18.844024599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:06.912433797 +0000 UTC m=+87.297922708" watchObservedRunningTime="2026-07-21 04:11:06.914426108 +0000 UTC m=+87.299915018"
Jul 21 04:11:06.993108 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:06.992391    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-75ljs" podStartSLOduration=68.073778857 podStartE2EDuration="1m26.992368153s" totalImagesPullingTime="18.918589296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:06.991087367 +0000 UTC m=+87.376576277" watchObservedRunningTime="2026-07-21 04:11:06.992368153 +0000 UTC m=+87.377857064"
Jul 21 04:11:07.857353 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:07.857227    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:07.962225 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:07.962160    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:08.808222 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:08.808169    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerStarted","Data":"3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b"}
Jul 21 04:11:08.813403 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:08.813345    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t" event={"ID":"b1f82002-4d5f-41b0-89dd-7673c1df3111","Type":"ContainerStarted","Data":"5000da43fa04cb3e3d7204bfe5baec6d14c5df049374b0d2c490b9cb254ae487"}
Jul 21 04:11:08.813607 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:08.813421    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t" event={"ID":"b1f82002-4d5f-41b0-89dd-7673c1df3111","Type":"ContainerStarted","Data":"cdca99459fe8549f4e8c7c726e8878b987b3038c6007ad5259311016df9e116e"}
Jul 21 04:11:08.813607 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:08.813462    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t" event={"ID":"b1f82002-4d5f-41b0-89dd-7673c1df3111","Type":"ContainerStarted","Data":"6be8f4cd28a1e189906b3b19854f37a76f70dc3a292061e179b260519068c28a"}
Jul 21 04:11:08.844131 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:08.843818    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.974424106 podStartE2EDuration="23.843796951s" totalImagesPullingTime="19.839872867s" totalInitContainerRuntime="29.499978ms" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:08.841428785 +0000 UTC m=+89.226917719" watchObservedRunningTime="2026-07-21 04:11:08.843796951 +0000 UTC m=+89.229285860"
Jul 21 04:11:08.851817 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:08.851790    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:11:08.852023 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:08.852011    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:11:08.867804 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:08.865508    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t" podStartSLOduration=3.232498472 podStartE2EDuration="22.865485143s" totalImagesPullingTime="19.632986671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:08.864738429 +0000 UTC m=+89.250227374" watchObservedRunningTime="2026-07-21 04:11:08.865485143 +0000 UTC m=+89.250974054"
Jul 21 04:11:09.819970 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:09.819939    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:11:10.842062 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:10.842031    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5884f7ddfb-twz9t"
Jul 21 04:11:11.845739 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:11.845701    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerStarted","Data":"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"}
Jul 21 04:11:12.860007 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:12.859966    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerStarted","Data":"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"}
Jul 21 04:11:12.860485 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:12.860084    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerStarted","Data":"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"}
Jul 21 04:11:12.860485 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:12.860134    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerStarted","Data":"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"}
Jul 21 04:11:12.860485 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:12.860176    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerStarted","Data":"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"}
Jul 21 04:11:12.860485 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:12.860203    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerStarted","Data":"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"}
Jul 21 04:11:12.890405 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:12.890346    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=18.036242968 podStartE2EDuration="22.890325248s" totalImagesPullingTime="4.81625428s" totalInitContainerRuntime="37.828ms" isStatefulPod=false podCreationTimestamp="2026-07-21 04:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:11:12.887766069 +0000 UTC m=+93.273255000" watchObservedRunningTime="2026-07-21 04:11:12.890325248 +0000 UTC m=+93.275814157"
Jul 21 04:11:14.421159 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:14.421096    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:14.522057 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:14.522023    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:15.822707 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:15.822663    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:11:16.209155 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:16.209052    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:11:19.108467 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:19.108430    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:19.162642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:19.162611    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:19.219720 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:19.219689    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:20.313854 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:20.313816    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:28.856885 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:28.856857    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:11:28.860663 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:28.860639    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7f44576f59-v5zpb"
Jul 21 04:11:31.814336 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:31.814256    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79f8bb54cc-g2pv8" podUID="b3c7d41a-71e2-4660-9ed1-b74b761886e5" containerName="console" containerID="cri-o://2865e948bca19532f6263a218b4277ed9a6c754585ef3453c54389c9b5a55918" gracePeriod=15
Jul 21 04:11:31.948028 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:31.947992    2546 generic.go:356] "Generic (PLEG): container finished" podID="b3c7d41a-71e2-4660-9ed1-b74b761886e5" containerID="2865e948bca19532f6263a218b4277ed9a6c754585ef3453c54389c9b5a55918" exitCode=0
Jul 21 04:11:31.948204 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:31.948070    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79f8bb54cc-g2pv8" event={"ID":"b3c7d41a-71e2-4660-9ed1-b74b761886e5","Type":"ContainerDied","Data":"2865e948bca19532f6263a218b4277ed9a6c754585ef3453c54389c9b5a55918"}
Jul 21 04:11:32.111757 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.111731    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:11:32.171247 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.171218    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-oauth-serving-cert\") pod \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") "
Jul 21 04:11:32.171473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.171289    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-service-ca\") pod \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") "
Jul 21 04:11:32.171473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.171335    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-oauth-config\") pod \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") "
Jul 21 04:11:32.171473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.171360    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-config\") pod \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") "
Jul 21 04:11:32.171473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.171379    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-serving-cert\") pod \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") "
Jul 21 04:11:32.171473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.171431    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpjw9\" (UniqueName: \"kubernetes.io/projected/b3c7d41a-71e2-4660-9ed1-b74b761886e5-kube-api-access-bpjw9\") pod \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\" (UID: \"b3c7d41a-71e2-4660-9ed1-b74b761886e5\") "
Jul 21 04:11:32.171752 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.171672    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b3c7d41a-71e2-4660-9ed1-b74b761886e5" (UID: "b3c7d41a-71e2-4660-9ed1-b74b761886e5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:11:32.171806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.171779    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-config" (OuterVolumeSpecName: "console-config") pod "b3c7d41a-71e2-4660-9ed1-b74b761886e5" (UID: "b3c7d41a-71e2-4660-9ed1-b74b761886e5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:11:32.172012 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.171973    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-service-ca" (OuterVolumeSpecName: "service-ca") pod "b3c7d41a-71e2-4660-9ed1-b74b761886e5" (UID: "b3c7d41a-71e2-4660-9ed1-b74b761886e5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:11:32.173930 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.173903    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b3c7d41a-71e2-4660-9ed1-b74b761886e5" (UID: "b3c7d41a-71e2-4660-9ed1-b74b761886e5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:11:32.174391 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.174369    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b3c7d41a-71e2-4660-9ed1-b74b761886e5" (UID: "b3c7d41a-71e2-4660-9ed1-b74b761886e5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:11:32.174481 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.174402    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3c7d41a-71e2-4660-9ed1-b74b761886e5-kube-api-access-bpjw9" (OuterVolumeSpecName: "kube-api-access-bpjw9") pod "b3c7d41a-71e2-4660-9ed1-b74b761886e5" (UID: "b3c7d41a-71e2-4660-9ed1-b74b761886e5"). InnerVolumeSpecName "kube-api-access-bpjw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:11:32.272798 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.272766    2546 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:11:32.272798 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.272797    2546 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-oauth-config\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:11:32.272993 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.272808    2546 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-config\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:11:32.272993 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.272817    2546 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c7d41a-71e2-4660-9ed1-b74b761886e5-console-serving-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:11:32.272993 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.272826    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpjw9\" (UniqueName: \"kubernetes.io/projected/b3c7d41a-71e2-4660-9ed1-b74b761886e5-kube-api-access-bpjw9\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:11:32.272993 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.272834    2546 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c7d41a-71e2-4660-9ed1-b74b761886e5-oauth-serving-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:11:32.954413 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.954381    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79f8bb54cc-g2pv8" event={"ID":"b3c7d41a-71e2-4660-9ed1-b74b761886e5","Type":"ContainerDied","Data":"5b31cf4cc031c32ae3a2581835747081b312334deb6abdaf8d33766a75c91cca"}
Jul 21 04:11:32.954825 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.954444    2546 scope.go:117] "RemoveContainer" containerID="2865e948bca19532f6263a218b4277ed9a6c754585ef3453c54389c9b5a55918"
Jul 21 04:11:32.954825 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.954440    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79f8bb54cc-g2pv8"
Jul 21 04:11:32.980611 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.980582    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79f8bb54cc-g2pv8"]
Jul 21 04:11:32.985586 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:32.985561    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79f8bb54cc-g2pv8"]
Jul 21 04:11:34.164689 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:34.164642    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3c7d41a-71e2-4660-9ed1-b74b761886e5" path="/var/lib/kubelet/pods/b3c7d41a-71e2-4660-9ed1-b74b761886e5/volumes"
Jul 21 04:11:38.817268 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:38.817239    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xv2m6"
Jul 21 04:11:46.981242 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:46.981197    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:11:50.822109 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:50.822062    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:11:50.877716 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:50.877684    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:11:51.051400 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:11:51.051373    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:00.237781 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:00.237747    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:00.290891 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:00.290860    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:04.486151 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:04.486098    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 04:12:04.486604 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:04.486426    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="alertmanager" containerID="cri-o://f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b" gracePeriod=120
Jul 21 04:12:04.486604 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:04.486556    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy-web" containerID="cri-o://1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c" gracePeriod=120
Jul 21 04:12:04.486729 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:04.486626    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy-metric" containerID="cri-o://935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef" gracePeriod=120
Jul 21 04:12:04.486729 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:04.486689    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="config-reloader" containerID="cri-o://a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e" gracePeriod=120
Jul 21 04:12:04.487425 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:04.486891    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy" containerID="cri-o://b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6" gracePeriod=120
Jul 21 04:12:04.487425 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:04.486891    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="prom-label-proxy" containerID="cri-o://3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b" gracePeriod=120
Jul 21 04:12:05.094766 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094731    2546 generic.go:356] "Generic (PLEG): container finished" podID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerID="3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b" exitCode=0
Jul 21 04:12:05.094766 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094756    2546 generic.go:356] "Generic (PLEG): container finished" podID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerID="935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef" exitCode=0
Jul 21 04:12:05.094766 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094762    2546 generic.go:356] "Generic (PLEG): container finished" podID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerID="b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6" exitCode=0
Jul 21 04:12:05.094766 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094770    2546 generic.go:356] "Generic (PLEG): container finished" podID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerID="a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e" exitCode=0
Jul 21 04:12:05.094766 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094775    2546 generic.go:356] "Generic (PLEG): container finished" podID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerID="f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b" exitCode=0
Jul 21 04:12:05.095055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094799    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerDied","Data":"3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b"}
Jul 21 04:12:05.095055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094859    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerDied","Data":"935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef"}
Jul 21 04:12:05.095055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094886    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerDied","Data":"b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6"}
Jul 21 04:12:05.095055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094910    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerDied","Data":"a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e"}
Jul 21 04:12:05.095055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.094934    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerDied","Data":"f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b"}
Jul 21 04:12:05.745035 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.745006    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:05.771372 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771320    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-metrics-client-ca\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.771526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771386    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-volume\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.771526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771426    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86dgz\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-kube-api-access-86dgz\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.771526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771459    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.771526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771486    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-main-db\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.771526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771519    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.772207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771555    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-cluster-tls-config\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.772207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771597    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-out\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.772207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771633    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-trusted-ca-bundle\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.772207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771655    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.772207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771677    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-web-config\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.772207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771696    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-tls-assets\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.772207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771729    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-main-tls\") pod \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\" (UID: \"0a6cdb45-0c1b-4355-a698-321b7c2be6b0\") "
Jul 21 04:12:05.772207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.771738    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:12:05.772207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.772037    2546 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-metrics-client-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.775023 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.773250    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 04:12:05.775023 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.774269    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:12:05.778849 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.777600    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:05.793828 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.793520    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:12:05.794048 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.794016    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-kube-api-access-86dgz" (OuterVolumeSpecName: "kube-api-access-86dgz") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "kube-api-access-86dgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:12:05.794530 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.794440    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-volume" (OuterVolumeSpecName: "config-volume") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:05.794652 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.794604    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-out" (OuterVolumeSpecName: "config-out") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 04:12:05.795013 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.794854    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:05.795266 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.795192    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:05.795359 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.795320    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:05.797638 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.797616    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:05.805550 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.805528    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-web-config" (OuterVolumeSpecName: "web-config") pod "0a6cdb45-0c1b-4355-a698-321b7c2be6b0" (UID: "0a6cdb45-0c1b-4355-a698-321b7c2be6b0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:05.872537 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872466    2546 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872537 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872493    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86dgz\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-kube-api-access-86dgz\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872537 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872504    2546 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872537 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872523    2546 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-main-db\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872537 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872532    2546 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872537 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872542    2546 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-cluster-tls-config\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872550    2546 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-config-out\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872558    2546 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872566    2546 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872593    2546 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-web-config\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872601    2546 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-tls-assets\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:05.872816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:05.872609    2546 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a6cdb45-0c1b-4355-a698-321b7c2be6b0-secret-alertmanager-main-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:06.101704 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.101666    2546 generic.go:356] "Generic (PLEG): container finished" podID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerID="1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c" exitCode=0
Jul 21 04:12:06.101936 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.101751    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerDied","Data":"1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c"}
Jul 21 04:12:06.101936 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.101771    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.101936 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.101845    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a6cdb45-0c1b-4355-a698-321b7c2be6b0","Type":"ContainerDied","Data":"31dff09f5dedea8501d18fc582e677db3ad392c586bdb6c73425dc0a0671cf24"}
Jul 21 04:12:06.101936 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.101879    2546 scope.go:117] "RemoveContainer" containerID="3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b"
Jul 21 04:12:06.110556 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.110534    2546 scope.go:117] "RemoveContainer" containerID="935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef"
Jul 21 04:12:06.118353 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.118335    2546 scope.go:117] "RemoveContainer" containerID="b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6"
Jul 21 04:12:06.125323 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.125286    2546 scope.go:117] "RemoveContainer" containerID="1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c"
Jul 21 04:12:06.127803 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.127735    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 04:12:06.133353 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.133337    2546 scope.go:117] "RemoveContainer" containerID="a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e"
Jul 21 04:12:06.134543 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.134525    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 04:12:06.140809 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.140791    2546 scope.go:117] "RemoveContainer" containerID="f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b"
Jul 21 04:12:06.147965 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.147948    2546 scope.go:117] "RemoveContainer" containerID="3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723"
Jul 21 04:12:06.155086 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.155071    2546 scope.go:117] "RemoveContainer" containerID="3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b"
Jul 21 04:12:06.155370 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:06.155347    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b\": container with ID starting with 3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b not found: ID does not exist" containerID="3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b"
Jul 21 04:12:06.155461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.155385    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b"} err="failed to get container status \"3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b\": rpc error: code = NotFound desc = could not find container \"3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b\": container with ID starting with 3da9eb5173ab34eab52cfdbf63a4aa3dcc51833608a121c2755f8bb04b01af7b not found: ID does not exist"
Jul 21 04:12:06.155461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.155431    2546 scope.go:117] "RemoveContainer" containerID="935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef"
Jul 21 04:12:06.155696 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:06.155676    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef\": container with ID starting with 935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef not found: ID does not exist" containerID="935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef"
Jul 21 04:12:06.155736 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.155703    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef"} err="failed to get container status \"935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef\": rpc error: code = NotFound desc = could not find container \"935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef\": container with ID starting with 935a7e69030cbf7ce487893156a97f15d1905c8e72babdc59ab4f5e0e88b44ef not found: ID does not exist"
Jul 21 04:12:06.155736 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.155719    2546 scope.go:117] "RemoveContainer" containerID="b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6"
Jul 21 04:12:06.155946 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:06.155924    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6\": container with ID starting with b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6 not found: ID does not exist" containerID="b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6"
Jul 21 04:12:06.155985 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.155951    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6"} err="failed to get container status \"b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6\": rpc error: code = NotFound desc = could not find container \"b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6\": container with ID starting with b92a867e388c5a718849b323294ececa6e1b618e6005fb5161f06640643319d6 not found: ID does not exist"
Jul 21 04:12:06.155985 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.155965    2546 scope.go:117] "RemoveContainer" containerID="1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c"
Jul 21 04:12:06.156208 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:06.156192    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c\": container with ID starting with 1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c not found: ID does not exist" containerID="1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c"
Jul 21 04:12:06.156257 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.156212    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c"} err="failed to get container status \"1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c\": rpc error: code = NotFound desc = could not find container \"1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c\": container with ID starting with 1c528112912979081436327346dfd692d744d9ca5baf33499835dbdfa4de872c not found: ID does not exist"
Jul 21 04:12:06.156257 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.156225    2546 scope.go:117] "RemoveContainer" containerID="a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e"
Jul 21 04:12:06.156454 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:06.156439    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e\": container with ID starting with a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e not found: ID does not exist" containerID="a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e"
Jul 21 04:12:06.156494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.156457    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e"} err="failed to get container status \"a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e\": rpc error: code = NotFound desc = could not find container \"a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e\": container with ID starting with a10ccd86ea978cd34e45c36bbb957e3710eeda9ba01227ff9ad4b84f8584ca8e not found: ID does not exist"
Jul 21 04:12:06.156494 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.156469    2546 scope.go:117] "RemoveContainer" containerID="f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b"
Jul 21 04:12:06.156693 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:06.156676    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b\": container with ID starting with f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b not found: ID does not exist" containerID="f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b"
Jul 21 04:12:06.156735 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.156695    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b"} err="failed to get container status \"f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b\": rpc error: code = NotFound desc = could not find container \"f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b\": container with ID starting with f8de3f6dd4593df3beedf1af77c4fe3c978a501e433a27ca28a511057689760b not found: ID does not exist"
Jul 21 04:12:06.156735 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.156707    2546 scope.go:117] "RemoveContainer" containerID="3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723"
Jul 21 04:12:06.156972 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:06.156944    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723\": container with ID starting with 3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723 not found: ID does not exist" containerID="3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723"
Jul 21 04:12:06.157071 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.156980    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723"} err="failed to get container status \"3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723\": rpc error: code = NotFound desc = could not find container \"3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723\": container with ID starting with 3d6696ceae7286506172f2684f756fcfef2a00f605b98728a50497dad951b723 not found: ID does not exist"
Jul 21 04:12:06.162330 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.162309    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" path="/var/lib/kubelet/pods/0a6cdb45-0c1b-4355-a698-321b7c2be6b0/volumes"
Jul 21 04:12:06.162786 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.162772    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 04:12:06.163070 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163059    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="config-reloader"
Jul 21 04:12:06.163105 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163072    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="config-reloader"
Jul 21 04:12:06.163105 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163079    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy-web"
Jul 21 04:12:06.163105 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163085    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy-web"
Jul 21 04:12:06.163105 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163092    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="prom-label-proxy"
Jul 21 04:12:06.163105 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163097    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="prom-label-proxy"
Jul 21 04:12:06.163105 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163103    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="init-config-reloader"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163108    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="init-config-reloader"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163145    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="alertmanager"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163151    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="alertmanager"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163156    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy-metric"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163161    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy-metric"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163169    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3c7d41a-71e2-4660-9ed1-b74b761886e5" containerName="console"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163173    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3c7d41a-71e2-4660-9ed1-b74b761886e5" containerName="console"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163185    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163190    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163251    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="config-reloader"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163259    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="prom-label-proxy"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163266    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy-metric"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163272    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163277    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3c7d41a-71e2-4660-9ed1-b74b761886e5" containerName="console"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163281    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="alertmanager"
Jul 21 04:12:06.163305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.163288    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6cdb45-0c1b-4355-a698-321b7c2be6b0" containerName="kube-rbac-proxy-web"
Jul 21 04:12:06.167596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.167581    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.170273 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.170249    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 21 04:12:06.170379 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.170361    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 21 04:12:06.170446 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.170434    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 21 04:12:06.170731 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.170719    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 21 04:12:06.170980 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.170939    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 21 04:12:06.170980 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.170962    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x5czt\""
Jul 21 04:12:06.171200 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.171185    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 21 04:12:06.171200 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.171195    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 21 04:12:06.171321 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.171190    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 21 04:12:06.176502 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.176483    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 21 04:12:06.179357 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.179336    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 04:12:06.276306 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276267    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24552c9b-7be4-4e35-a861-ae6b668cd04e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276306 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276303    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276335    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276422    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-config-volume\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276457    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276484    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8bnc\" (UniqueName: \"kubernetes.io/projected/24552c9b-7be4-4e35-a861-ae6b668cd04e-kube-api-access-h8bnc\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276514 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276509    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/24552c9b-7be4-4e35-a861-ae6b668cd04e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276667 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276524    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-web-config\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276667 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276540    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/24552c9b-7be4-4e35-a861-ae6b668cd04e-config-out\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276667 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276555    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276667 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276581    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276667 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276595    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/24552c9b-7be4-4e35-a861-ae6b668cd04e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.276667 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.276612    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24552c9b-7be4-4e35-a861-ae6b668cd04e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377272 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377191    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-config-volume\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377272 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377235    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377272 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377260    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8bnc\" (UniqueName: \"kubernetes.io/projected/24552c9b-7be4-4e35-a861-ae6b668cd04e-kube-api-access-h8bnc\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377506 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377297    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/24552c9b-7be4-4e35-a861-ae6b668cd04e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377506 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377322    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-web-config\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377506 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377362    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/24552c9b-7be4-4e35-a861-ae6b668cd04e-config-out\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377506 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377387    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377506 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377422    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377506 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377445    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/24552c9b-7be4-4e35-a861-ae6b668cd04e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377506 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377473    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24552c9b-7be4-4e35-a861-ae6b668cd04e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377523    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24552c9b-7be4-4e35-a861-ae6b668cd04e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377552    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.377810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.377642    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.379007 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.378022    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/24552c9b-7be4-4e35-a861-ae6b668cd04e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.379007 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.378927    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24552c9b-7be4-4e35-a861-ae6b668cd04e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.379383 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.379360    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24552c9b-7be4-4e35-a861-ae6b668cd04e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.380919 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.380585    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-config-volume\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.380919 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.380802    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.380919 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.380873    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.380919 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.380887    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.381208 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.380932    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.381208 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.381080    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/24552c9b-7be4-4e35-a861-ae6b668cd04e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.381602 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.381585    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/24552c9b-7be4-4e35-a861-ae6b668cd04e-config-out\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.382041 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.382022    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-web-config\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.382355 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.382337    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/24552c9b-7be4-4e35-a861-ae6b668cd04e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.386019 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.386002    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8bnc\" (UniqueName: \"kubernetes.io/projected/24552c9b-7be4-4e35-a861-ae6b668cd04e-kube-api-access-h8bnc\") pod \"alertmanager-main-0\" (UID: \"24552c9b-7be4-4e35-a861-ae6b668cd04e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.477434 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.477397    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 04:12:06.620224 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:06.620196    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 04:12:06.624427 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:12:06.624397    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24552c9b_7be4_4e35_a861_ae6b668cd04e.slice/crio-47dfbe344ec4444c479b04490e35f522c420fa9f244342d202c0b631bc64fa5a WatchSource:0}: Error finding container 47dfbe344ec4444c479b04490e35f522c420fa9f244342d202c0b631bc64fa5a: Status 404 returned error can't find the container with id 47dfbe344ec4444c479b04490e35f522c420fa9f244342d202c0b631bc64fa5a
Jul 21 04:12:07.108475 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:07.108440    2546 generic.go:356] "Generic (PLEG): container finished" podID="24552c9b-7be4-4e35-a861-ae6b668cd04e" containerID="9ce977f0bc567299a1d44144eee93bf549ee21cb7427450f048a8f9e706956df" exitCode=0
Jul 21 04:12:07.108475 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:07.108479    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"24552c9b-7be4-4e35-a861-ae6b668cd04e","Type":"ContainerDied","Data":"9ce977f0bc567299a1d44144eee93bf549ee21cb7427450f048a8f9e706956df"}
Jul 21 04:12:07.108995 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:07.108528    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"24552c9b-7be4-4e35-a861-ae6b668cd04e","Type":"ContainerStarted","Data":"47dfbe344ec4444c479b04490e35f522c420fa9f244342d202c0b631bc64fa5a"}
Jul 21 04:12:08.116194 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.116160    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"24552c9b-7be4-4e35-a861-ae6b668cd04e","Type":"ContainerStarted","Data":"6735d7fc75d10c17be65e2f0ece106694a7d2b0e8c4a21dde7c2c22b0175c47f"}
Jul 21 04:12:08.116566 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.116246    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"24552c9b-7be4-4e35-a861-ae6b668cd04e","Type":"ContainerStarted","Data":"3b5a81c62d58c9457e5427a3778cad4a3e45c081712d9376af64f186268c6a1d"}
Jul 21 04:12:08.116566 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.116278    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"24552c9b-7be4-4e35-a861-ae6b668cd04e","Type":"ContainerStarted","Data":"4efb1d2e1f695e61953041d2a9a5b469746d2a615c4b2648041ecc5e4eea9b46"}
Jul 21 04:12:08.116566 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.116302    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"24552c9b-7be4-4e35-a861-ae6b668cd04e","Type":"ContainerStarted","Data":"3be80f76d03de88a5a9e1c14df937404a454bb31b140a5fd42ead34f7e8e3903"}
Jul 21 04:12:08.116566 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.116325    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"24552c9b-7be4-4e35-a861-ae6b668cd04e","Type":"ContainerStarted","Data":"73687b73438919830b7a141e7a64190531c795f21d234e40b46caa66145c05da"}
Jul 21 04:12:08.116566 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.116349    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"24552c9b-7be4-4e35-a861-ae6b668cd04e","Type":"ContainerStarted","Data":"8a70d22ac2a80d5ecfe2b3ae4ed6221be7d953cc900a58d36976fc1a94f427a1"}
Jul 21 04:12:08.142526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.142482    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.12631284 podStartE2EDuration="2.142468724s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.155884ms" isStatefulPod=false podCreationTimestamp="2026-07-21 04:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:12:08.141013572 +0000 UTC m=+148.526502497" watchObservedRunningTime="2026-07-21 04:12:08.142468724 +0000 UTC m=+148.527957636"
Jul 21 04:12:08.719170 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.719138    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 04:12:08.719558 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.719454    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="prometheus" containerID="cri-o://23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6" gracePeriod=600
Jul 21 04:12:08.719558 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.719484    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662" gracePeriod=600
Jul 21 04:12:08.719558 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.719510    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="thanos-sidecar" containerID="cri-o://fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda" gracePeriod=600
Jul 21 04:12:08.719558 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.719518    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="config-reloader" containerID="cri-o://d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672" gracePeriod=600
Jul 21 04:12:08.719921 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.719512    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy-web" containerID="cri-o://dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5" gracePeriod=600
Jul 21 04:12:08.719921 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.719458    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy" containerID="cri-o://747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a" gracePeriod=600
Jul 21 04:12:08.988880 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:08.988855    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.043106 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.043072    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:09.102369 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102339    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-config-out\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.102369 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102376    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-0\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.102620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102400    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-web-config\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.102620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102418    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-kubelet-serving-ca-bundle\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.102620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102442    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-tls-assets\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.102620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102468    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-serving-certs-ca-bundle\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.102620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102496    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.102620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102525    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-thanos-prometheus-http-client-file\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.102620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102555    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-trusted-ca-bundle\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.102620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102582    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-db\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102638    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-tls\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102675    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-grpc-tls\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102702    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-config\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102732    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-metrics-client-certs\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102767    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-kube-rbac-proxy\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102787    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102812    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102837    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9zjv\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-kube-api-access-r9zjv\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102847    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102873    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-1\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102910    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-2\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.102941    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-metrics-client-ca\") pod \"46eeae26-1bda-4a50-8430-1cd5487760b0\" (UID: \"46eeae26-1bda-4a50-8430-1cd5487760b0\") "
Jul 21 04:12:09.103768 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.103216    2546 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.103768 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.103235    2546 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.103768 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.103587    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:12:09.103935 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.103870    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:12:09.104050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.103974    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:12:09.105776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.104290    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:12:09.105776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.105739    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 04:12:09.106499 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.106475    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:12:09.106890 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.106854    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:09.106890 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.106787    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:12:09.107455 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.107428    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-kube-api-access-r9zjv" (OuterVolumeSpecName: "kube-api-access-r9zjv") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "kube-api-access-r9zjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:12:09.107552 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.107462    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:09.107552 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.107508    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-config" (OuterVolumeSpecName: "config") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:09.107699 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.107576    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:09.107767 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.107737    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:09.107893 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.107785    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:09.108052 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.108033    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-config-out" (OuterVolumeSpecName: "config-out") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 04:12:09.108151 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.108135    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:09.108737 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.108716    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:09.117409 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.117382    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-web-config" (OuterVolumeSpecName: "web-config") pod "46eeae26-1bda-4a50-8430-1cd5487760b0" (UID: "46eeae26-1bda-4a50-8430-1cd5487760b0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:12:09.124473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124449    2546 generic.go:356] "Generic (PLEG): container finished" podID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerID="e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662" exitCode=0
Jul 21 04:12:09.124473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124473    2546 generic.go:356] "Generic (PLEG): container finished" podID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerID="747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a" exitCode=0
Jul 21 04:12:09.124615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124483    2546 generic.go:356] "Generic (PLEG): container finished" podID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerID="dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5" exitCode=0
Jul 21 04:12:09.124615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124491    2546 generic.go:356] "Generic (PLEG): container finished" podID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerID="fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda" exitCode=0
Jul 21 04:12:09.124615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124499    2546 generic.go:356] "Generic (PLEG): container finished" podID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerID="d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672" exitCode=0
Jul 21 04:12:09.124615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124507    2546 generic.go:356] "Generic (PLEG): container finished" podID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerID="23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6" exitCode=0
Jul 21 04:12:09.124615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124532    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.124615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124532    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerDied","Data":"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"}
Jul 21 04:12:09.124615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124603    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerDied","Data":"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"}
Jul 21 04:12:09.124810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124635    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerDied","Data":"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"}
Jul 21 04:12:09.124810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124664    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerDied","Data":"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"}
Jul 21 04:12:09.124810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124693    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerDied","Data":"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"}
Jul 21 04:12:09.124810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124696    2546 scope.go:117] "RemoveContainer" containerID="e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"
Jul 21 04:12:09.124810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124719    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerDied","Data":"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"}
Jul 21 04:12:09.124810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.124752    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46eeae26-1bda-4a50-8430-1cd5487760b0","Type":"ContainerDied","Data":"827816675f02d2e1b5c7fbc17e63a0afeb007b0ca9dbfcfa30c43c42dd94f28a"}
Jul 21 04:12:09.137886 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.137863    2546 scope.go:117] "RemoveContainer" containerID="747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"
Jul 21 04:12:09.145305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.145289    2546 scope.go:117] "RemoveContainer" containerID="dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"
Jul 21 04:12:09.146520 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.146498    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:09.152444 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.152420    2546 scope.go:117] "RemoveContainer" containerID="fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"
Jul 21 04:12:09.155635 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.155610    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 04:12:09.159029 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.159008    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 04:12:09.161342 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.161325    2546 scope.go:117] "RemoveContainer" containerID="d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"
Jul 21 04:12:09.168470 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.168452    2546 scope.go:117] "RemoveContainer" containerID="23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"
Jul 21 04:12:09.176158 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.176141    2546 scope.go:117] "RemoveContainer" containerID="cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"
Jul 21 04:12:09.183422 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.183349    2546 scope.go:117] "RemoveContainer" containerID="e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"
Jul 21 04:12:09.183755 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:09.183722    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": container with ID starting with e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662 not found: ID does not exist" containerID="e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"
Jul 21 04:12:09.183824 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.183759    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"} err="failed to get container status \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": rpc error: code = NotFound desc = could not find container \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": container with ID starting with e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662 not found: ID does not exist"
Jul 21 04:12:09.183824 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.183784    2546 scope.go:117] "RemoveContainer" containerID="747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"
Jul 21 04:12:09.184101 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:09.184081    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": container with ID starting with 747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a not found: ID does not exist" containerID="747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"
Jul 21 04:12:09.184199 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.184106    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"} err="failed to get container status \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": rpc error: code = NotFound desc = could not find container \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": container with ID starting with 747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a not found: ID does not exist"
Jul 21 04:12:09.184199 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.184148    2546 scope.go:117] "RemoveContainer" containerID="dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"
Jul 21 04:12:09.184421 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:09.184397    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": container with ID starting with dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5 not found: ID does not exist" containerID="dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"
Jul 21 04:12:09.184475 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.184429    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"} err="failed to get container status \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": rpc error: code = NotFound desc = could not find container \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": container with ID starting with dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5 not found: ID does not exist"
Jul 21 04:12:09.184475 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.184447    2546 scope.go:117] "RemoveContainer" containerID="fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"
Jul 21 04:12:09.184698 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:09.184683    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": container with ID starting with fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda not found: ID does not exist" containerID="fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"
Jul 21 04:12:09.184740 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.184701    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"} err="failed to get container status \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": rpc error: code = NotFound desc = could not find container \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": container with ID starting with fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda not found: ID does not exist"
Jul 21 04:12:09.184740 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.184714    2546 scope.go:117] "RemoveContainer" containerID="d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"
Jul 21 04:12:09.184961 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:09.184938    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": container with ID starting with d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672 not found: ID does not exist" containerID="d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"
Jul 21 04:12:09.185004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.184971    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"} err="failed to get container status \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": rpc error: code = NotFound desc = could not find container \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": container with ID starting with d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672 not found: ID does not exist"
Jul 21 04:12:09.185004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.184991    2546 scope.go:117] "RemoveContainer" containerID="23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"
Jul 21 04:12:09.185229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185210    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 04:12:09.185277 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:09.185231    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": container with ID starting with 23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6 not found: ID does not exist" containerID="23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"
Jul 21 04:12:09.185277 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185249    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"} err="failed to get container status \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": rpc error: code = NotFound desc = could not find container \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": container with ID starting with 23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6 not found: ID does not exist"
Jul 21 04:12:09.185277 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185260    2546 scope.go:117] "RemoveContainer" containerID="cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"
Jul 21 04:12:09.185483 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:12:09.185467    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": container with ID starting with cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37 not found: ID does not exist" containerID="cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"
Jul 21 04:12:09.185521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185485    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"} err="failed to get container status \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": rpc error: code = NotFound desc = could not find container \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": container with ID starting with cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37 not found: ID does not exist"
Jul 21 04:12:09.185521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185497    2546 scope.go:117] "RemoveContainer" containerID="e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"
Jul 21 04:12:09.185601 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185555    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy-thanos"
Jul 21 04:12:09.185601 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185568    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy-thanos"
Jul 21 04:12:09.185601 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185582    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="prometheus"
Jul 21 04:12:09.185601 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185587    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="prometheus"
Jul 21 04:12:09.185601 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185597    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy-web"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185607    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy-web"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185620    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="config-reloader"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185628    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="config-reloader"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185647    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="init-config-reloader"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185655    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="init-config-reloader"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185662    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="thanos-sidecar"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185667    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="thanos-sidecar"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185663    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"} err="failed to get container status \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": rpc error: code = NotFound desc = could not find container \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": container with ID starting with e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662 not found: ID does not exist"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185681    2546 scope.go:117] "RemoveContainer" containerID="747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185674    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy"
Jul 21 04:12:09.185756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185715    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185797    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="config-reloader"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185807    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="thanos-sidecar"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185819    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185829    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy-thanos"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185838    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="prometheus"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185844    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" containerName="kube-rbac-proxy-web"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185858    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"} err="failed to get container status \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": rpc error: code = NotFound desc = could not find container \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": container with ID starting with 747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a not found: ID does not exist"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.185872    2546 scope.go:117] "RemoveContainer" containerID="dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186048    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"} err="failed to get container status \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": rpc error: code = NotFound desc = could not find container \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": container with ID starting with dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5 not found: ID does not exist"
Jul 21 04:12:09.186234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186077    2546 scope.go:117] "RemoveContainer" containerID="fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"
Jul 21 04:12:09.186666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186324    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"} err="failed to get container status \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": rpc error: code = NotFound desc = could not find container \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": container with ID starting with fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda not found: ID does not exist"
Jul 21 04:12:09.186666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186346    2546 scope.go:117] "RemoveContainer" containerID="d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"
Jul 21 04:12:09.186666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186512    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"} err="failed to get container status \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": rpc error: code = NotFound desc = could not find container \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": container with ID starting with d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672 not found: ID does not exist"
Jul 21 04:12:09.186666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186532    2546 scope.go:117] "RemoveContainer" containerID="23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"
Jul 21 04:12:09.186803 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186695    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"} err="failed to get container status \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": rpc error: code = NotFound desc = could not find container \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": container with ID starting with 23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6 not found: ID does not exist"
Jul 21 04:12:09.186803 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186708    2546 scope.go:117] "RemoveContainer" containerID="cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"
Jul 21 04:12:09.186942 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186924    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"} err="failed to get container status \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": rpc error: code = NotFound desc = could not find container \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": container with ID starting with cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37 not found: ID does not exist"
Jul 21 04:12:09.186978 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.186946    2546 scope.go:117] "RemoveContainer" containerID="e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"
Jul 21 04:12:09.187202 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.187186    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"} err="failed to get container status \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": rpc error: code = NotFound desc = could not find container \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": container with ID starting with e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662 not found: ID does not exist"
Jul 21 04:12:09.187258 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.187203    2546 scope.go:117] "RemoveContainer" containerID="747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"
Jul 21 04:12:09.187454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.187440    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"} err="failed to get container status \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": rpc error: code = NotFound desc = could not find container \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": container with ID starting with 747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a not found: ID does not exist"
Jul 21 04:12:09.187454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.187453    2546 scope.go:117] "RemoveContainer" containerID="dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"
Jul 21 04:12:09.187635 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.187621    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"} err="failed to get container status \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": rpc error: code = NotFound desc = could not find container \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": container with ID starting with dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5 not found: ID does not exist"
Jul 21 04:12:09.187635 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.187634    2546 scope.go:117] "RemoveContainer" containerID="fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"
Jul 21 04:12:09.187808 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.187794    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"} err="failed to get container status \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": rpc error: code = NotFound desc = could not find container \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": container with ID starting with fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda not found: ID does not exist"
Jul 21 04:12:09.187808 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.187807    2546 scope.go:117] "RemoveContainer" containerID="d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"
Jul 21 04:12:09.188043 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.188023    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"} err="failed to get container status \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": rpc error: code = NotFound desc = could not find container \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": container with ID starting with d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672 not found: ID does not exist"
Jul 21 04:12:09.188043 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.188043    2546 scope.go:117] "RemoveContainer" containerID="23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"
Jul 21 04:12:09.188325 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.188297    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"} err="failed to get container status \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": rpc error: code = NotFound desc = could not find container \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": container with ID starting with 23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6 not found: ID does not exist"
Jul 21 04:12:09.188427 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.188326    2546 scope.go:117] "RemoveContainer" containerID="cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"
Jul 21 04:12:09.188655 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.188631    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"} err="failed to get container status \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": rpc error: code = NotFound desc = could not find container \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": container with ID starting with cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37 not found: ID does not exist"
Jul 21 04:12:09.188655 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.188655    2546 scope.go:117] "RemoveContainer" containerID="e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"
Jul 21 04:12:09.188918 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.188886    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"} err="failed to get container status \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": rpc error: code = NotFound desc = could not find container \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": container with ID starting with e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662 not found: ID does not exist"
Jul 21 04:12:09.188918 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.188907    2546 scope.go:117] "RemoveContainer" containerID="747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"
Jul 21 04:12:09.189188 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.189171    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"} err="failed to get container status \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": rpc error: code = NotFound desc = could not find container \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": container with ID starting with 747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a not found: ID does not exist"
Jul 21 04:12:09.189188 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.189188    2546 scope.go:117] "RemoveContainer" containerID="dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"
Jul 21 04:12:09.189423 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.189403    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"} err="failed to get container status \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": rpc error: code = NotFound desc = could not find container \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": container with ID starting with dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5 not found: ID does not exist"
Jul 21 04:12:09.189478 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.189424    2546 scope.go:117] "RemoveContainer" containerID="fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"
Jul 21 04:12:09.189640 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.189622    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"} err="failed to get container status \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": rpc error: code = NotFound desc = could not find container \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": container with ID starting with fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda not found: ID does not exist"
Jul 21 04:12:09.189688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.189641    2546 scope.go:117] "RemoveContainer" containerID="d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"
Jul 21 04:12:09.190155 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.190133    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"} err="failed to get container status \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": rpc error: code = NotFound desc = could not find container \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": container with ID starting with d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672 not found: ID does not exist"
Jul 21 04:12:09.190204 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.190157    2546 scope.go:117] "RemoveContainer" containerID="23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"
Jul 21 04:12:09.190260 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.190232    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.190745 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.190451    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"} err="failed to get container status \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": rpc error: code = NotFound desc = could not find container \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": container with ID starting with 23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6 not found: ID does not exist"
Jul 21 04:12:09.190745 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.190481    2546 scope.go:117] "RemoveContainer" containerID="cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"
Jul 21 04:12:09.190885 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.190799    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"} err="failed to get container status \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": rpc error: code = NotFound desc = could not find container \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": container with ID starting with cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37 not found: ID does not exist"
Jul 21 04:12:09.190885 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.190823    2546 scope.go:117] "RemoveContainer" containerID="e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"
Jul 21 04:12:09.191047 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191025    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"} err="failed to get container status \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": rpc error: code = NotFound desc = could not find container \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": container with ID starting with e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662 not found: ID does not exist"
Jul 21 04:12:09.191086 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191049    2546 scope.go:117] "RemoveContainer" containerID="747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"
Jul 21 04:12:09.191300 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191285    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"} err="failed to get container status \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": rpc error: code = NotFound desc = could not find container \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": container with ID starting with 747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a not found: ID does not exist"
Jul 21 04:12:09.191347 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191301    2546 scope.go:117] "RemoveContainer" containerID="dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"
Jul 21 04:12:09.191523 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191507    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"} err="failed to get container status \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": rpc error: code = NotFound desc = could not find container \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": container with ID starting with dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5 not found: ID does not exist"
Jul 21 04:12:09.191561 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191523    2546 scope.go:117] "RemoveContainer" containerID="fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"
Jul 21 04:12:09.191734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191718    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"} err="failed to get container status \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": rpc error: code = NotFound desc = could not find container \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": container with ID starting with fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda not found: ID does not exist"
Jul 21 04:12:09.191778 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191736    2546 scope.go:117] "RemoveContainer" containerID="d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"
Jul 21 04:12:09.191934 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191916    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"} err="failed to get container status \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": rpc error: code = NotFound desc = could not find container \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": container with ID starting with d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672 not found: ID does not exist"
Jul 21 04:12:09.191984 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.191933    2546 scope.go:117] "RemoveContainer" containerID="23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"
Jul 21 04:12:09.192147 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.192093    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"} err="failed to get container status \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": rpc error: code = NotFound desc = could not find container \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": container with ID starting with 23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6 not found: ID does not exist"
Jul 21 04:12:09.192229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.192160    2546 scope.go:117] "RemoveContainer" containerID="cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"
Jul 21 04:12:09.192412 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.192395    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"} err="failed to get container status \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": rpc error: code = NotFound desc = could not find container \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": container with ID starting with cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37 not found: ID does not exist"
Jul 21 04:12:09.192451 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.192415    2546 scope.go:117] "RemoveContainer" containerID="e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"
Jul 21 04:12:09.192718 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.192693    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662"} err="failed to get container status \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": rpc error: code = NotFound desc = could not find container \"e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662\": container with ID starting with e2f3ad4404e956c0a2059f6d45908c9646ae41bd1a9a135a5a181a169e11f662 not found: ID does not exist"
Jul 21 04:12:09.192795 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.192720    2546 scope.go:117] "RemoveContainer" containerID="747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"
Jul 21 04:12:09.193008 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.192982    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a"} err="failed to get container status \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": rpc error: code = NotFound desc = could not find container \"747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a\": container with ID starting with 747beab77a42aeccddcadb9d64e8852c486ba4d893a7cd661a4e03557df09e4a not found: ID does not exist"
Jul 21 04:12:09.193133 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193009    2546 scope.go:117] "RemoveContainer" containerID="dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"
Jul 21 04:12:09.193202 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193096    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5060cv17crf5t\""
Jul 21 04:12:09.193202 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193183    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 21 04:12:09.193332 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193308    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5"} err="failed to get container status \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": rpc error: code = NotFound desc = could not find container \"dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5\": container with ID starting with dc097f96e0d3c51ed733e92c877cf92ab80f4f8be3fdfbac50bf516d0ab81de5 not found: ID does not exist"
Jul 21 04:12:09.193411 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193335    2546 scope.go:117] "RemoveContainer" containerID="fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"
Jul 21 04:12:09.193485 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193412    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 21 04:12:09.193485 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193466    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 21 04:12:09.193624 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193492    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 21 04:12:09.193624 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193496    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 21 04:12:09.193624 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193562    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda"} err="failed to get container status \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": rpc error: code = NotFound desc = could not find container \"fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda\": container with ID starting with fca64ca7a72d2257c7b09b61f757b419da9a32d524d9087e6b58a3bb89806dda not found: ID does not exist"
Jul 21 04:12:09.193624 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193584    2546 scope.go:117] "RemoveContainer" containerID="d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"
Jul 21 04:12:09.193624 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193617    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 21 04:12:09.193861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193806    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672"} err="failed to get container status \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": rpc error: code = NotFound desc = could not find container \"d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672\": container with ID starting with d38f0281f9457df5fafa9851d776d963105cebc999256319b257d56cdafa4672 not found: ID does not exist"
Jul 21 04:12:09.193861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.193827    2546 scope.go:117] "RemoveContainer" containerID="23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"
Jul 21 04:12:09.194073 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194050    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6"} err="failed to get container status \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": rpc error: code = NotFound desc = could not find container \"23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6\": container with ID starting with 23ed473694ecefb361b98484ead833b9bf282ac853ca955e066798622f17aed6 not found: ID does not exist"
Jul 21 04:12:09.194148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194074    2546 scope.go:117] "RemoveContainer" containerID="cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"
Jul 21 04:12:09.194336 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194212    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 21 04:12:09.194336 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194220    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 21 04:12:09.194336 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194279    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 21 04:12:09.194336 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194303    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37"} err="failed to get container status \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": rpc error: code = NotFound desc = could not find container \"cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37\": container with ID starting with cd4281c167887527b4134a7c4fef914f74d7177939286a42065afa1247999c37 not found: ID does not exist"
Jul 21 04:12:09.194510 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194354    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 21 04:12:09.194510 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194502    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2pkdp\""
Jul 21 04:12:09.194638 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194624    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 21 04:12:09.194694 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.194681    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 21 04:12:09.197773 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.197755    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 21 04:12:09.199592 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.199573    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204526    2546 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204552    2546 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-configmap-metrics-client-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204568    2546 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-config-out\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204582    2546 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204595    2546 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-web-config\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204608    2546 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-tls-assets\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204622    2546 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204638    2546 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204653    2546 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204666    2546 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-db\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204681    2546 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204695    2546 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-grpc-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204707    2546 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-config\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204720    2546 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-metrics-client-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204733    2546 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-kube-rbac-proxy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204747    2546 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46eeae26-1bda-4a50-8430-1cd5487760b0-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.204781    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9zjv\" (UniqueName: \"kubernetes.io/projected/46eeae26-1bda-4a50-8430-1cd5487760b0-kube-api-access-r9zjv\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.205154 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.205093    2546 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46eeae26-1bda-4a50-8430-1cd5487760b0-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:12:09.206972 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.206885    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 04:12:09.306097 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306005    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-config\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306097 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306043    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306078    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306192    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhp6x\" (UniqueName: \"kubernetes.io/projected/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-kube-api-access-fhp6x\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306224    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306255    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306281    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306304    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306332    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306374    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306434    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-config-out\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306462    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306479    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306509    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306524    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306541    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306560    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306579    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306909 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306621    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-web-config\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.306909 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.306640    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.407516 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407484    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-config\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.407670 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407522    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.407670 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407554    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.407670 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407571    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhp6x\" (UniqueName: \"kubernetes.io/projected/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-kube-api-access-fhp6x\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.407670 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407588    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.407670 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407607    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.407670 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407624    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.407670 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407664    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407700    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407733    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407769    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-config-out\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407803    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407834    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407893    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407923    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407950    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.407982    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.408521    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.408740    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.409228    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.409590    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.409922    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.410047    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.410098    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-web-config\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.411851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.410146    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.412710 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.410283    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.412710 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.410830    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.412710 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.410948    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.412710 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.411324    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-config\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.412710 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.411763    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.412710 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.411789    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.412710 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.412369    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.412710 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.412609    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.413102 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.412910    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.413102 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.413093    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-config-out\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.413350 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.413330    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.413462 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.413445    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.413694 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.413675    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-web-config\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.413895 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.413879    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.416818 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.416802    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhp6x\" (UniqueName: \"kubernetes.io/projected/c43ef906-877b-44ef-bc7b-6ecaaafb6da8-kube-api-access-fhp6x\") pod \"prometheus-k8s-0\" (UID: \"c43ef906-877b-44ef-bc7b-6ecaaafb6da8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.501990 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.501953    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:09.642386 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:09.642362    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 04:12:09.643941 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:12:09.643920    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43ef906_877b_44ef_bc7b_6ecaaafb6da8.slice/crio-354624974bbc776d4f39af8bde2faef181a2174b35092a4601cd6fa52ae845b3 WatchSource:0}: Error finding container 354624974bbc776d4f39af8bde2faef181a2174b35092a4601cd6fa52ae845b3: Status 404 returned error can't find the container with id 354624974bbc776d4f39af8bde2faef181a2174b35092a4601cd6fa52ae845b3
Jul 21 04:12:10.131791 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:10.131757    2546 generic.go:356] "Generic (PLEG): container finished" podID="c43ef906-877b-44ef-bc7b-6ecaaafb6da8" containerID="c4a9a63d8dbcdf28978dbba278cd75e7d01c6f0b6b7dbde659ad662f4e336ee3" exitCode=0
Jul 21 04:12:10.132225 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:10.131853    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c43ef906-877b-44ef-bc7b-6ecaaafb6da8","Type":"ContainerDied","Data":"c4a9a63d8dbcdf28978dbba278cd75e7d01c6f0b6b7dbde659ad662f4e336ee3"}
Jul 21 04:12:10.132225 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:10.131928    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c43ef906-877b-44ef-bc7b-6ecaaafb6da8","Type":"ContainerStarted","Data":"354624974bbc776d4f39af8bde2faef181a2174b35092a4601cd6fa52ae845b3"}
Jul 21 04:12:10.169415 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:10.169386    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46eeae26-1bda-4a50-8430-1cd5487760b0" path="/var/lib/kubelet/pods/46eeae26-1bda-4a50-8430-1cd5487760b0/volumes"
Jul 21 04:12:11.140783 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:11.140747    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c43ef906-877b-44ef-bc7b-6ecaaafb6da8","Type":"ContainerStarted","Data":"4114fe73a962490a99377ff500e8c5331ecb57cfc691f723325fea7f506a1231"}
Jul 21 04:12:11.141259 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:11.140812    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c43ef906-877b-44ef-bc7b-6ecaaafb6da8","Type":"ContainerStarted","Data":"72bb17c7c8858b72edad2ff7b4a2407d9d0dddf8af4a7a46be5cbae64bb17470"}
Jul 21 04:12:11.141259 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:11.140837    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c43ef906-877b-44ef-bc7b-6ecaaafb6da8","Type":"ContainerStarted","Data":"5cfeed252e91f31e3bcee24375f62813a3b0affaaeaeff98d34bc40c1a7411d0"}
Jul 21 04:12:11.141259 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:11.140863    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c43ef906-877b-44ef-bc7b-6ecaaafb6da8","Type":"ContainerStarted","Data":"d903998b9e01c45c255db033c6dfe27afd6257afd4152f00569cb853771ee0b2"}
Jul 21 04:12:11.141259 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:11.140885    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c43ef906-877b-44ef-bc7b-6ecaaafb6da8","Type":"ContainerStarted","Data":"91f77fc62f6db9009918a1d0b863a48a8da8f66854f25cde9fe97ff3d2555d87"}
Jul 21 04:12:11.141259 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:11.140907    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c43ef906-877b-44ef-bc7b-6ecaaafb6da8","Type":"ContainerStarted","Data":"183a5260a4a842908fc85570e0e264a03b45aa8deb2a111c9a5e9c9b8b6eb989"}
Jul 21 04:12:11.170163 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:11.170094    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.1488970370000002 podStartE2EDuration="2.170079144s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.182107ms" isStatefulPod=false podCreationTimestamp="2026-07-21 04:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:12:11.168822695 +0000 UTC m=+151.554311625" watchObservedRunningTime="2026-07-21 04:12:11.170079144 +0000 UTC m=+151.555568052"
Jul 21 04:12:12.135583 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:12.135542    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:12.190470 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:12.190420    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:14.502537 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:14.502488    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:12:20.352854 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:20.352821    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:20.453258 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:20.453225    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:20.506414 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:20.506381    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:29.422319 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:29.422284    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:29.524322 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:29.524294    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:52.040318 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:52.040282    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:52.104977 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:52.104945    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:12:58.395513 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:12:58.395482    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:06.473191 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:06.473159    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:06.568797 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:06.568763    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:09.502530 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:09.502497    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:13:09.518519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:09.518494    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:13:10.391600 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:10.391571    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 04:13:28.081435 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:28.081403    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:28.896411 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:28.896371    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:31.709729 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:31.709696    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:31.809761 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:31.809726    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:32.891017 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:32.890987    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:32.987473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:32.987444    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:33.172766 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:33.172690    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:33.341295 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:33.341261    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:34.896714 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:34.896680    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:35.448186 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.448148    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sv2vc"]
Jul 21 04:13:35.450488 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.450473    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:35.453520 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.453500    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 21 04:13:35.454704 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.454685    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 21 04:13:35.454803 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.454728    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qqtd8\""
Jul 21 04:13:35.464064 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.464041    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sv2vc"]
Jul 21 04:13:35.525134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.525088    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:35.532469 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.532443    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/354cc865-5be6-4245-9f08-db4e982d4ae8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sv2vc\" (UID: \"354cc865-5be6-4245-9f08-db4e982d4ae8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:35.532612 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.532502    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhrjs\" (UniqueName: \"kubernetes.io/projected/354cc865-5be6-4245-9f08-db4e982d4ae8-kube-api-access-dhrjs\") pod \"cert-manager-webhook-6486ffd997-sv2vc\" (UID: \"354cc865-5be6-4245-9f08-db4e982d4ae8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:35.633530 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.633492    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhrjs\" (UniqueName: \"kubernetes.io/projected/354cc865-5be6-4245-9f08-db4e982d4ae8-kube-api-access-dhrjs\") pod \"cert-manager-webhook-6486ffd997-sv2vc\" (UID: \"354cc865-5be6-4245-9f08-db4e982d4ae8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:35.633698 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.633580    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/354cc865-5be6-4245-9f08-db4e982d4ae8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sv2vc\" (UID: \"354cc865-5be6-4245-9f08-db4e982d4ae8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:35.642719 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.642687    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhrjs\" (UniqueName: \"kubernetes.io/projected/354cc865-5be6-4245-9f08-db4e982d4ae8-kube-api-access-dhrjs\") pod \"cert-manager-webhook-6486ffd997-sv2vc\" (UID: \"354cc865-5be6-4245-9f08-db4e982d4ae8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:35.642719 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.642698    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/354cc865-5be6-4245-9f08-db4e982d4ae8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sv2vc\" (UID: \"354cc865-5be6-4245-9f08-db4e982d4ae8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:35.772371 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.772341    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:35.916706 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:35.916665    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sv2vc"]
Jul 21 04:13:35.919399 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:13:35.919367    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod354cc865_5be6_4245_9f08_db4e982d4ae8.slice/crio-fdc8b379fce8f59eab8f1e40af5c373b91014a22703d10167158197258041648 WatchSource:0}: Error finding container fdc8b379fce8f59eab8f1e40af5c373b91014a22703d10167158197258041648: Status 404 returned error can't find the container with id fdc8b379fce8f59eab8f1e40af5c373b91014a22703d10167158197258041648
Jul 21 04:13:36.481180 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:36.481148    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc" event={"ID":"354cc865-5be6-4245-9f08-db4e982d4ae8","Type":"ContainerStarted","Data":"fdc8b379fce8f59eab8f1e40af5c373b91014a22703d10167158197258041648"}
Jul 21 04:13:38.488169 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:38.488133    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:38.557252 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:38.557199    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:38.619248 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:38.619210    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:38.684063 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:38.684024    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:39.498218 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:39.498187    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc" event={"ID":"354cc865-5be6-4245-9f08-db4e982d4ae8","Type":"ContainerStarted","Data":"c7f4d9a9b2e64ab8f867e4f506e00327a58453c932513257eecad5815aceeded"}
Jul 21 04:13:39.498700 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:39.498398    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:39.516407 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:39.516358    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc" podStartSLOduration=1.2530418349999999 podStartE2EDuration="4.516341825s" totalImagesPullingTime="3.26329999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:13:39.514420968 +0000 UTC m=+239.899909877" watchObservedRunningTime="2026-07-21 04:13:39.516341825 +0000 UTC m=+239.901830733"
Jul 21 04:13:41.936993 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:41.936937    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:42.038272 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:42.038239    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:43.416152 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.416104    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:43.514865 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.514833    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:43.913071 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.913038    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"]
Jul 21 04:13:43.915473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.915457    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"
Jul 21 04:13:43.918136 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.918089    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 21 04:13:43.918257 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.918097    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-6pgbv\""
Jul 21 04:13:43.918993 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.918976    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 21 04:13:43.924237 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.924215    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"]
Jul 21 04:13:43.988547 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.988519    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88c9f72f-8c5d-418c-93a7-44f4b53e4a99-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6htbb\" (UID: \"88c9f72f-8c5d-418c-93a7-44f4b53e4a99\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"
Jul 21 04:13:43.988547 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:43.988550    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwh97\" (UniqueName: \"kubernetes.io/projected/88c9f72f-8c5d-418c-93a7-44f4b53e4a99-kube-api-access-gwh97\") pod \"openshift-lws-operator-fd8ccff4c-6htbb\" (UID: \"88c9f72f-8c5d-418c-93a7-44f4b53e4a99\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"
Jul 21 04:13:44.088980 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:44.088937    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88c9f72f-8c5d-418c-93a7-44f4b53e4a99-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6htbb\" (UID: \"88c9f72f-8c5d-418c-93a7-44f4b53e4a99\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"
Jul 21 04:13:44.088980 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:44.088985    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwh97\" (UniqueName: \"kubernetes.io/projected/88c9f72f-8c5d-418c-93a7-44f4b53e4a99-kube-api-access-gwh97\") pod \"openshift-lws-operator-fd8ccff4c-6htbb\" (UID: \"88c9f72f-8c5d-418c-93a7-44f4b53e4a99\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"
Jul 21 04:13:44.089418 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:44.089395    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88c9f72f-8c5d-418c-93a7-44f4b53e4a99-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6htbb\" (UID: \"88c9f72f-8c5d-418c-93a7-44f4b53e4a99\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"
Jul 21 04:13:44.098060 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:44.098038    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwh97\" (UniqueName: \"kubernetes.io/projected/88c9f72f-8c5d-418c-93a7-44f4b53e4a99-kube-api-access-gwh97\") pod \"openshift-lws-operator-fd8ccff4c-6htbb\" (UID: \"88c9f72f-8c5d-418c-93a7-44f4b53e4a99\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"
Jul 21 04:13:44.225272 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:44.225248    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"
Jul 21 04:13:44.299834 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:44.299779    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:44.362441 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:44.362406    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb"]
Jul 21 04:13:44.365177 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:13:44.365156    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88c9f72f_8c5d_418c_93a7_44f4b53e4a99.slice/crio-2059598c1fdccaf3d7a3b34b432522b325f901ec808abd26d80827625f5d5977 WatchSource:0}: Error finding container 2059598c1fdccaf3d7a3b34b432522b325f901ec808abd26d80827625f5d5977: Status 404 returned error can't find the container with id 2059598c1fdccaf3d7a3b34b432522b325f901ec808abd26d80827625f5d5977
Jul 21 04:13:44.518528 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:44.518438    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb" event={"ID":"88c9f72f-8c5d-418c-93a7-44f4b53e4a99","Type":"ContainerStarted","Data":"2059598c1fdccaf3d7a3b34b432522b325f901ec808abd26d80827625f5d5977"}
Jul 21 04:13:45.504283 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:45.504252    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sv2vc"
Jul 21 04:13:47.539555 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:47.539524    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb" event={"ID":"88c9f72f-8c5d-418c-93a7-44f4b53e4a99","Type":"ContainerStarted","Data":"e8dbb7788ca52aa373dabe878b0875685b973ae78e991c8bccc61c04384e81a6"}
Jul 21 04:13:47.557198 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:47.557149    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6htbb" podStartSLOduration=2.385563022 podStartE2EDuration="4.557133618s" totalImagesPullingTime="2.171570596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:13:47.556619908 +0000 UTC m=+247.942108828" watchObservedRunningTime="2026-07-21 04:13:47.557133618 +0000 UTC m=+247.942622527"
Jul 21 04:13:48.228401 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:48.228368    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:48.332382 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:48.332350    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:13:50.318083 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:13:50.318049    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:07.155464 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:07.155427    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:07.252956 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:07.252921    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:07.707392 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:07.707353    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:07.910596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:07.910563    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:08.014824 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.014794    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:08.091473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.091444    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"]
Jul 21 04:14:08.101562 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.101531    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.104280 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.104258    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 04:14:08.104412 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.104258    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sznh7\""
Jul 21 04:14:08.104646 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.104546    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 04:14:08.104646 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.104560    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 04:14:08.104646 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.104607    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 04:14:08.107440 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.107419    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"]
Jul 21 04:14:08.185553 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.185528    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-tmp\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.185989 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.185576    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-manifests\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.185989 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.185596    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.185989 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.185724    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.185989 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.185761    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrht5\" (UniqueName: \"kubernetes.io/projected/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-kube-api-access-vrht5\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.216726 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.216691    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:08.286584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.286519    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.286584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.286549    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrht5\" (UniqueName: \"kubernetes.io/projected/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-kube-api-access-vrht5\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.286584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.286582    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-tmp\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.286783 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.286621    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-manifests\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.286783 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.286639    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.287098 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.287077    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-manifests\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.289009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.288987    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-tmp\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.289162 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.289145    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.289234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.289216    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.295169 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.295148    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrht5\" (UniqueName: \"kubernetes.io/projected/f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4-kube-api-access-vrht5\") pod \"opendatahub-operator-controller-manager-7d877cbcc8-5br7f\" (UID: \"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.317500 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.317477    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:08.413773 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.413748    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:08.423285 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.423256    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:08.518734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.518697    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:08.554191 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.554109    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"]
Jul 21 04:14:08.557827 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:14:08.557801    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf739ef16_adf9_41d0_b0c0_e8c7ebb2e6b4.slice/crio-4a778cf65c01a7ff8fca63fb3df4fbfda7ec00f426396a9dd93f76f545e45825 WatchSource:0}: Error finding container 4a778cf65c01a7ff8fca63fb3df4fbfda7ec00f426396a9dd93f76f545e45825: Status 404 returned error can't find the container with id 4a778cf65c01a7ff8fca63fb3df4fbfda7ec00f426396a9dd93f76f545e45825
Jul 21 04:14:08.627614 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.627590    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f" event={"ID":"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4","Type":"ContainerStarted","Data":"4a778cf65c01a7ff8fca63fb3df4fbfda7ec00f426396a9dd93f76f545e45825"}
Jul 21 04:14:08.632349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:08.632327    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:11.645805 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:11.645772    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f" event={"ID":"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4","Type":"ContainerStarted","Data":"5fb13520364cf6c95c2731260f3bac2bddc82742b07db10a15febf65f9ebf16e"}
Jul 21 04:14:12.651635 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:12.651605    2546 generic.go:356] "Generic (PLEG): container finished" podID="f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4" containerID="5fb13520364cf6c95c2731260f3bac2bddc82742b07db10a15febf65f9ebf16e" exitCode=0
Jul 21 04:14:12.652018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:12.651690    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f" event={"ID":"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4","Type":"ContainerDied","Data":"5fb13520364cf6c95c2731260f3bac2bddc82742b07db10a15febf65f9ebf16e"}
Jul 21 04:14:13.658930 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:13.658895    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f" event={"ID":"f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4","Type":"ContainerStarted","Data":"4d02f9d5c8de057edf3b532242d6ac8bb26275760886583a0b07b8efafa314ab"}
Jul 21 04:14:13.659446 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:13.659042    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:13.682926 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:13.682885    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f" podStartSLOduration=1.410407591 podStartE2EDuration="5.682871359s" totalImagesPullingTime="3.197352797s" totalInitContainerRuntime="1.075110971s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:14:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:14:13.681439829 +0000 UTC m=+274.066928740" watchObservedRunningTime="2026-07-21 04:14:13.682871359 +0000 UTC m=+274.068360268"
Jul 21 04:14:18.141453 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:18.141422    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:19.330345 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:19.330311    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:19.901003 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:19.900966    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:20.002501 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:20.002478    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:21.447085 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:21.447056    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:21.625273 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:21.625238    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:24.665102 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:24.665071    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d877cbcc8-5br7f"
Jul 21 04:14:26.654587 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:26.654553    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:26.746632 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:26.746598    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:26.948051 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:26.947959    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:27.048451 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:27.048423    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:27.410947 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:27.410919    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:27.511564 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:27.511532    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:27.611665 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:27.611638    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:27.714826 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:27.714747    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:28.220667 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:28.220635    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:28.462162 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:28.462109    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:28.533048 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:28.532954    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:28.779865 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:28.779820    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:29.095846 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:29.095815    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:29.503567 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:29.503528    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:32.985771 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:32.985730    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:33.080281 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:33.080244    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:33.617578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:33.617545    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:33.768985 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:33.768944    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:33.905614 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:33.905531    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:35.327070 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.327012    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"]
Jul 21 04:14:35.330807 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.330787    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.339693 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.339672    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 21 04:14:35.339787 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.339672    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 21 04:14:35.339787 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.339723    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 21 04:14:35.340505 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.340490    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vpxss\""
Jul 21 04:14:35.354066 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.354046    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"]
Jul 21 04:14:35.439055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.438975    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-metrics-cert\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.439055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.439032    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5xhk\" (UniqueName: \"kubernetes.io/projected/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-kube-api-access-k5xhk\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.439257 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.439187    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-manager-config\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.439257 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.439216    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-cert\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.539677 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.539646    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-manager-config\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.539677 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.539679    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-cert\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.539906 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.539706    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-metrics-cert\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.539906 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.539834    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5xhk\" (UniqueName: \"kubernetes.io/projected/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-kube-api-access-k5xhk\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.540612 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.540586    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-manager-config\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.543042 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.543004    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-cert\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.543593 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.543572    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-metrics-cert\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.551813 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.551794    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5xhk\" (UniqueName: \"kubernetes.io/projected/deaf9dfa-9998-4a2a-9898-fedf4ffbb784-kube-api-access-k5xhk\") pod \"lws-controller-manager-b844b6c94-c79g4\" (UID: \"deaf9dfa-9998-4a2a-9898-fedf4ffbb784\") " pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.639637 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.639603    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:35.686211 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.686177    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"]
Jul 21 04:14:35.689837 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.689790    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.692574 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.692550    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 04:14:35.692705 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.692642    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 04:14:35.692964 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.692946    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 04:14:35.693061 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.693026    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7qbwr\""
Jul 21 04:14:35.693216 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.693199    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 04:14:35.693292 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.693231    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 04:14:35.693292 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.693233    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 04:14:35.709134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.705855    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"]
Jul 21 04:14:35.742030 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.741998    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.742187 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.742048    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.742187 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.742174    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/791657e0-9d92-4cbc-8804-be0d29845b9a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.742314 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.742215    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t88qd\" (UniqueName: \"kubernetes.io/projected/791657e0-9d92-4cbc-8804-be0d29845b9a-kube-api-access-t88qd\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.742358 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.742324    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.742358 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.742348    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/791657e0-9d92-4cbc-8804-be0d29845b9a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.742457 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.742427    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.787921 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.787892    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"]
Jul 21 04:14:35.792040 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:14:35.792010    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeaf9dfa_9998_4a2a_9898_fedf4ffbb784.slice/crio-99a38c67b4d8742c2c18159741692eabc545f7d76bfeaf21f07c3fc572b9daa2 WatchSource:0}: Error finding container 99a38c67b4d8742c2c18159741692eabc545f7d76bfeaf21f07c3fc572b9daa2: Status 404 returned error can't find the container with id 99a38c67b4d8742c2c18159741692eabc545f7d76bfeaf21f07c3fc572b9daa2
Jul 21 04:14:35.843678 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.843644    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.843678 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.843686    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.843882 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.843863    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/791657e0-9d92-4cbc-8804-be0d29845b9a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.843920 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.843899    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t88qd\" (UniqueName: \"kubernetes.io/projected/791657e0-9d92-4cbc-8804-be0d29845b9a-kube-api-access-t88qd\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.843983 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.843969    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.844028 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.843998    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/791657e0-9d92-4cbc-8804-be0d29845b9a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.844082 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.844050    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.844549 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.844519    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.846398 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.846369    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/791657e0-9d92-4cbc-8804-be0d29845b9a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.846398 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.846383    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.846541 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.846524    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/791657e0-9d92-4cbc-8804-be0d29845b9a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.846579 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.846551    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.852883 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.852860    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t88qd\" (UniqueName: \"kubernetes.io/projected/791657e0-9d92-4cbc-8804-be0d29845b9a-kube-api-access-t88qd\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:35.853403 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:35.853378    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/791657e0-9d92-4cbc-8804-be0d29845b9a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hv9vk\" (UID: \"791657e0-9d92-4cbc-8804-be0d29845b9a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:36.010281 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:36.010246    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:36.179306 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:36.179274    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"]
Jul 21 04:14:36.179790 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:14:36.179769    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod791657e0_9d92_4cbc_8804_be0d29845b9a.slice/crio-ca449c9f2df945f1273c7d96f05fbe008beee915fa76d4422e272aa668f8cb2c WatchSource:0}: Error finding container ca449c9f2df945f1273c7d96f05fbe008beee915fa76d4422e272aa668f8cb2c: Status 404 returned error can't find the container with id ca449c9f2df945f1273c7d96f05fbe008beee915fa76d4422e272aa668f8cb2c
Jul 21 04:14:36.774887 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:36.774811    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:36.779385 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:36.779295    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4" event={"ID":"deaf9dfa-9998-4a2a-9898-fedf4ffbb784","Type":"ContainerStarted","Data":"99a38c67b4d8742c2c18159741692eabc545f7d76bfeaf21f07c3fc572b9daa2"}
Jul 21 04:14:36.784242 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:36.784175    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk" event={"ID":"791657e0-9d92-4cbc-8804-be0d29845b9a","Type":"ContainerStarted","Data":"ca449c9f2df945f1273c7d96f05fbe008beee915fa76d4422e272aa668f8cb2c"}
Jul 21 04:14:36.872500 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:36.872359    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:36.988427 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:36.988393    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:37.087778 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:37.087700    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:37.216885 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:37.216848    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:37.336128 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:37.336088    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:37.791544 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:37.791506    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4" event={"ID":"deaf9dfa-9998-4a2a-9898-fedf4ffbb784","Type":"ContainerStarted","Data":"6bd1f3085b88e1fdad376d8c18d194d892e0580105dd05b573bfaf68b321c60e"}
Jul 21 04:14:37.791936 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:37.791656    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:14:37.811454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:37.811404    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4" podStartSLOduration=1.007655928 podStartE2EDuration="2.811386931s" totalImagesPullingTime="1.803731003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:14:37.809797745 +0000 UTC m=+298.195286657" watchObservedRunningTime="2026-07-21 04:14:37.811386931 +0000 UTC m=+298.196875840"
Jul 21 04:14:38.587694 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:38.587654    2546 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 04:14:38.587855 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:38.587757    2546 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 04:14:38.798293 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:38.798254    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk" event={"ID":"791657e0-9d92-4cbc-8804-be0d29845b9a","Type":"ContainerStarted","Data":"76d49fdbb511efde81e33355af7826450d950fb3c76c0d2cbbaf8dd4a86182d0"}
Jul 21 04:14:38.798735 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:38.798369    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:38.823397 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:38.823342    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk" podStartSLOduration=1.417801896 podStartE2EDuration="3.823315116s" totalImagesPullingTime="2.40551322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:14:38.82192727 +0000 UTC m=+299.207416203" watchObservedRunningTime="2026-07-21 04:14:38.823315116 +0000 UTC m=+299.208804026"
Jul 21 04:14:39.805145 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:39.805103    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hv9vk"
Jul 21 04:14:40.004174 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:40.004149    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:14:40.004698 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:40.004670    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:14:40.011872 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:40.011851    2546 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 04:14:43.717395 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:43.717364    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:43.786375 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:43.786349    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:44.440500 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:44.440468    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:44.504583 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:44.504546    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:14:48.800807 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:14:48.800780    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-b844b6c94-c79g4"
Jul 21 04:15:28.124560 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:15:28.124518    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:15:28.223728 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:15:28.223694    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:15:28.297340 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:15:28.297306    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:15:58.052385 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:15:58.052351    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:03.239998 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:03.239959    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:06.434301 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:06.434265    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:07.228258 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:07.228225    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:10.064034 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:10.064002    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:10.163732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:10.163703    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:11.239342 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:11.239300    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:12.072744 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:12.072709    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:13.706801 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:13.706769    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:16.086207 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:16.086172    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:18.197463 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:18.197430    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:18.297399 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:18.297369    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:18.690484 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:18.690435    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:18.792484 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:18.792454    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:22.449771 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:22.449738    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:22.579213 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:22.579179    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:24.001991 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:24.001958    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:24.300851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:24.300770    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:26.292035 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:26.291999    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:26.354920 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:26.354884    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:26.436636 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:26.436604    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:26.503774 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:26.503741    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:29.606503 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:29.606466    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:29.710585 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:29.710553    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:29.812970 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:29.812934    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:29.914847 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:29.914768    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:34.089782 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.089743    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"]
Jul 21 04:16:34.092725 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.092702    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"
Jul 21 04:16:34.095355 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.095323    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 04:16:34.095456 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.095323    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 04:16:34.096461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.096439    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kkqmh\""
Jul 21 04:16:34.105941 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.105920    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"]
Jul 21 04:16:34.230392 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.230363    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s82z\" (UniqueName: \"kubernetes.io/projected/c1d9a61c-9e77-4203-b226-54e43c763fd6-kube-api-access-8s82z\") pod \"limitador-operator-controller-manager-859d7b78bd-l9cxb\" (UID: \"c1d9a61c-9e77-4203-b226-54e43c763fd6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"
Jul 21 04:16:34.331563 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.331521    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s82z\" (UniqueName: \"kubernetes.io/projected/c1d9a61c-9e77-4203-b226-54e43c763fd6-kube-api-access-8s82z\") pod \"limitador-operator-controller-manager-859d7b78bd-l9cxb\" (UID: \"c1d9a61c-9e77-4203-b226-54e43c763fd6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"
Jul 21 04:16:34.340728 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.340670    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s82z\" (UniqueName: \"kubernetes.io/projected/c1d9a61c-9e77-4203-b226-54e43c763fd6-kube-api-access-8s82z\") pod \"limitador-operator-controller-manager-859d7b78bd-l9cxb\" (UID: \"c1d9a61c-9e77-4203-b226-54e43c763fd6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"
Jul 21 04:16:34.403557 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.403529    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"
Jul 21 04:16:34.539263 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.539185    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"]
Jul 21 04:16:34.541612 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:16:34.541587    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d9a61c_9e77_4203_b226_54e43c763fd6.slice/crio-4c0b826dc032ab5d2a07bbef0c74de83acce0b2d247fd1d570cba34b58367d19 WatchSource:0}: Error finding container 4c0b826dc032ab5d2a07bbef0c74de83acce0b2d247fd1d570cba34b58367d19: Status 404 returned error can't find the container with id 4c0b826dc032ab5d2a07bbef0c74de83acce0b2d247fd1d570cba34b58367d19
Jul 21 04:16:34.543727 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:34.543713    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 04:16:35.357743 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:35.357713    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb" event={"ID":"c1d9a61c-9e77-4203-b226-54e43c763fd6","Type":"ContainerStarted","Data":"4c0b826dc032ab5d2a07bbef0c74de83acce0b2d247fd1d570cba34b58367d19"}
Jul 21 04:16:36.363816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:36.363783    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb" event={"ID":"c1d9a61c-9e77-4203-b226-54e43c763fd6","Type":"ContainerStarted","Data":"688de215858c195b0ec9fa726b6650a83a59cacc189d156bc92430292cf7d0f2"}
Jul 21 04:16:36.364185 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:36.363895    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"
Jul 21 04:16:36.384057 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:36.383994    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb" podStartSLOduration=0.67956623 podStartE2EDuration="2.383980154s" totalImagesPullingTime="1.704413924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:16:36.38270336 +0000 UTC m=+416.768192280" watchObservedRunningTime="2026-07-21 04:16:36.383980154 +0000 UTC m=+416.769469063"
Jul 21 04:16:36.555888 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:36.555804    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:36.656131 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:36.656095    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:39.681628 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:39.681588    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:39.780720 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:39.780692    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:44.286276 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:44.286246    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:44.348172 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:44.348144    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:44.475830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:44.475799    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:44.539217 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:44.539130    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:47.370714 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:47.370684    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9cxb"
Jul 21 04:16:49.279939 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:49.279903    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:16:49.344647 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:16:49.344619    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:14.022128 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:14.022073    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:14.207835 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:14.207805    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:27.795423 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:27.795392    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:27.886704 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:27.886658    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:27.947248 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:27.947221    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:28.667677 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:28.667638    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:28.773736 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:28.773706    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:29.425753 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:29.425723    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:29.522164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:29.522108    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:45.113296 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:45.113266    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:45.211761 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:45.211730    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:45.611450 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:45.611414    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:58.312478 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:58.312440    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:17:58.691150 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:17:58.691045    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:04.493181 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:04.493144    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:05.143801 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.143756    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:05.144658 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.144630    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d9f95f8cb-g9wcs"]
Jul 21 04:18:05.148149 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.148131    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:05.151832 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.151813    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4nl9r\""
Jul 21 04:18:05.151942 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.151858    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 04:18:05.152893 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.152876    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 04:18:05.160081 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.160059    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d9f95f8cb-g9wcs"]
Jul 21 04:18:05.208968 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.208925    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:05.210097 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.210074    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pvj5\" (UniqueName: \"kubernetes.io/projected/fcf3d664-9507-43f7-a584-1585cc4690d9-kube-api-access-4pvj5\") pod \"maas-controller-7d9f95f8cb-g9wcs\" (UID: \"fcf3d664-9507-43f7-a584-1585cc4690d9\") " pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:05.210263 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.210192    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcf3d664-9507-43f7-a584-1585cc4690d9-webhook-cert\") pod \"maas-controller-7d9f95f8cb-g9wcs\" (UID: \"fcf3d664-9507-43f7-a584-1585cc4690d9\") " pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:05.268683 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.268652    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:05.307521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.307485    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-rmstp"]
Jul 21 04:18:05.310907 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.310876    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pvj5\" (UniqueName: \"kubernetes.io/projected/fcf3d664-9507-43f7-a584-1585cc4690d9-kube-api-access-4pvj5\") pod \"maas-controller-7d9f95f8cb-g9wcs\" (UID: \"fcf3d664-9507-43f7-a584-1585cc4690d9\") " pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:05.311059 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.310950    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:05.311059 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.310958    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcf3d664-9507-43f7-a584-1585cc4690d9-webhook-cert\") pod \"maas-controller-7d9f95f8cb-g9wcs\" (UID: \"fcf3d664-9507-43f7-a584-1585cc4690d9\") " pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:05.311195 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:18:05.311091    2546 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 21 04:18:05.311195 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:18:05.311182    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf3d664-9507-43f7-a584-1585cc4690d9-webhook-cert podName:fcf3d664-9507-43f7-a584-1585cc4690d9 nodeName:}" failed. No retries permitted until 2026-07-21 04:18:05.811159072 +0000 UTC m=+506.196647973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/fcf3d664-9507-43f7-a584-1585cc4690d9-webhook-cert") pod "maas-controller-7d9f95f8cb-g9wcs" (UID: "fcf3d664-9507-43f7-a584-1585cc4690d9") : secret "maas-controller-webhook-cert" not found
Jul 21 04:18:05.321045 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.321019    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-rmstp"]
Jul 21 04:18:05.323610 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.323583    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pvj5\" (UniqueName: \"kubernetes.io/projected/fcf3d664-9507-43f7-a584-1585cc4690d9-kube-api-access-4pvj5\") pod \"maas-controller-7d9f95f8cb-g9wcs\" (UID: \"fcf3d664-9507-43f7-a584-1585cc4690d9\") " pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:05.411906 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.411818    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce1aa7c8-813e-4340-9486-ed68001ff6e6-webhook-cert\") pod \"maas-controller-6ffdf96fc7-rmstp\" (UID: \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\") " pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:05.411906 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.411856    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hw89\" (UniqueName: \"kubernetes.io/projected/ce1aa7c8-813e-4340-9486-ed68001ff6e6-kube-api-access-2hw89\") pod \"maas-controller-6ffdf96fc7-rmstp\" (UID: \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\") " pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:05.473204 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.473174    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:05.513298 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.513262    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce1aa7c8-813e-4340-9486-ed68001ff6e6-webhook-cert\") pod \"maas-controller-6ffdf96fc7-rmstp\" (UID: \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\") " pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:05.513298 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.513296    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hw89\" (UniqueName: \"kubernetes.io/projected/ce1aa7c8-813e-4340-9486-ed68001ff6e6-kube-api-access-2hw89\") pod \"maas-controller-6ffdf96fc7-rmstp\" (UID: \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\") " pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:05.515781 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.515759    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce1aa7c8-813e-4340-9486-ed68001ff6e6-webhook-cert\") pod \"maas-controller-6ffdf96fc7-rmstp\" (UID: \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\") " pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:05.526362 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.526334    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hw89\" (UniqueName: \"kubernetes.io/projected/ce1aa7c8-813e-4340-9486-ed68001ff6e6-kube-api-access-2hw89\") pod \"maas-controller-6ffdf96fc7-rmstp\" (UID: \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\") " pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:05.622475 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.622432    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:05.754414 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.754389    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-rmstp"]
Jul 21 04:18:05.756968 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:18:05.756943    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1aa7c8_813e_4340_9486_ed68001ff6e6.slice/crio-1e87aec6c9e89b6710f49fd05ea761164a675e2c875672433318b949043021d6 WatchSource:0}: Error finding container 1e87aec6c9e89b6710f49fd05ea761164a675e2c875672433318b949043021d6: Status 404 returned error can't find the container with id 1e87aec6c9e89b6710f49fd05ea761164a675e2c875672433318b949043021d6
Jul 21 04:18:05.789069 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.789036    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp" event={"ID":"ce1aa7c8-813e-4340-9486-ed68001ff6e6","Type":"ContainerStarted","Data":"1e87aec6c9e89b6710f49fd05ea761164a675e2c875672433318b949043021d6"}
Jul 21 04:18:05.816590 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.816560    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcf3d664-9507-43f7-a584-1585cc4690d9-webhook-cert\") pod \"maas-controller-7d9f95f8cb-g9wcs\" (UID: \"fcf3d664-9507-43f7-a584-1585cc4690d9\") " pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:05.819073 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:05.819056    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcf3d664-9507-43f7-a584-1585cc4690d9-webhook-cert\") pod \"maas-controller-7d9f95f8cb-g9wcs\" (UID: \"fcf3d664-9507-43f7-a584-1585cc4690d9\") " pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:06.058851 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:06.058800    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:06.190639 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:06.190595    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d9f95f8cb-g9wcs"]
Jul 21 04:18:06.192911 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:18:06.192885    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf3d664_9507_43f7_a584_1585cc4690d9.slice/crio-4b655967c66ac0ca0f20fc24fed36ab4e2fb226e4a8880b42f1057c17889cdcb WatchSource:0}: Error finding container 4b655967c66ac0ca0f20fc24fed36ab4e2fb226e4a8880b42f1057c17889cdcb: Status 404 returned error can't find the container with id 4b655967c66ac0ca0f20fc24fed36ab4e2fb226e4a8880b42f1057c17889cdcb
Jul 21 04:18:06.798519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:06.798447    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs" event={"ID":"fcf3d664-9507-43f7-a584-1585cc4690d9","Type":"ContainerStarted","Data":"4b655967c66ac0ca0f20fc24fed36ab4e2fb226e4a8880b42f1057c17889cdcb"}
Jul 21 04:18:08.514739 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:08.514708    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:08.609048 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:08.608978    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:08.812248 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:08.812210    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs" event={"ID":"fcf3d664-9507-43f7-a584-1585cc4690d9","Type":"ContainerStarted","Data":"c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07"}
Jul 21 04:18:08.812433 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:08.812340    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:08.813697 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:08.813671    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp" event={"ID":"ce1aa7c8-813e-4340-9486-ed68001ff6e6","Type":"ContainerStarted","Data":"5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305"}
Jul 21 04:18:08.813838 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:08.813723    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:08.850057 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:08.850011    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs" podStartSLOduration=1.7030827990000001 podStartE2EDuration="3.849998786s" totalImagesPullingTime="2.146915987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:18:08.848398835 +0000 UTC m=+509.233887756" watchObservedRunningTime="2026-07-21 04:18:08.849998786 +0000 UTC m=+509.235487695"
Jul 21 04:18:08.878575 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:08.878481    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp" podStartSLOduration=1.302828818 podStartE2EDuration="3.878466381s" totalImagesPullingTime="2.575637563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:18:08.875683167 +0000 UTC m=+509.261172079" watchObservedRunningTime="2026-07-21 04:18:08.878466381 +0000 UTC m=+509.263955289"
Jul 21 04:18:19.821172 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:19.821142    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:19.821572 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:19.821212    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:18:19.881172 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:19.881140    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d9f95f8cb-g9wcs"]
Jul 21 04:18:19.881353 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:19.881332    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs" podUID="fcf3d664-9507-43f7-a584-1585cc4690d9" containerName="manager" containerID="cri-o://c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07" gracePeriod=30
Jul 21 04:18:20.130938 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.130916    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:20.250633 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.250601    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pvj5\" (UniqueName: \"kubernetes.io/projected/fcf3d664-9507-43f7-a584-1585cc4690d9-kube-api-access-4pvj5\") pod \"fcf3d664-9507-43f7-a584-1585cc4690d9\" (UID: \"fcf3d664-9507-43f7-a584-1585cc4690d9\") "
Jul 21 04:18:20.250798 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.250675    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcf3d664-9507-43f7-a584-1585cc4690d9-webhook-cert\") pod \"fcf3d664-9507-43f7-a584-1585cc4690d9\" (UID: \"fcf3d664-9507-43f7-a584-1585cc4690d9\") "
Jul 21 04:18:20.253059 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.253024    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcf3d664-9507-43f7-a584-1585cc4690d9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fcf3d664-9507-43f7-a584-1585cc4690d9" (UID: "fcf3d664-9507-43f7-a584-1585cc4690d9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:18:20.253059 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.253031    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf3d664-9507-43f7-a584-1585cc4690d9-kube-api-access-4pvj5" (OuterVolumeSpecName: "kube-api-access-4pvj5") pod "fcf3d664-9507-43f7-a584-1585cc4690d9" (UID: "fcf3d664-9507-43f7-a584-1585cc4690d9"). InnerVolumeSpecName "kube-api-access-4pvj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:18:20.351603 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.351572    2546 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcf3d664-9507-43f7-a584-1585cc4690d9-webhook-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:18:20.351603 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.351600    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pvj5\" (UniqueName: \"kubernetes.io/projected/fcf3d664-9507-43f7-a584-1585cc4690d9-kube-api-access-4pvj5\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:18:20.873039 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.873001    2546 generic.go:356] "Generic (PLEG): container finished" podID="fcf3d664-9507-43f7-a584-1585cc4690d9" containerID="c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07" exitCode=0
Jul 21 04:18:20.873445 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.873065    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs"
Jul 21 04:18:20.873445 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.873087    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs" event={"ID":"fcf3d664-9507-43f7-a584-1585cc4690d9","Type":"ContainerDied","Data":"c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07"}
Jul 21 04:18:20.873445 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.873159    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9f95f8cb-g9wcs" event={"ID":"fcf3d664-9507-43f7-a584-1585cc4690d9","Type":"ContainerDied","Data":"4b655967c66ac0ca0f20fc24fed36ab4e2fb226e4a8880b42f1057c17889cdcb"}
Jul 21 04:18:20.873445 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.873182    2546 scope.go:117] "RemoveContainer" containerID="c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07"
Jul 21 04:18:20.882178 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.882016    2546 scope.go:117] "RemoveContainer" containerID="c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07"
Jul 21 04:18:20.882313 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:18:20.882293    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07\": container with ID starting with c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07 not found: ID does not exist" containerID="c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07"
Jul 21 04:18:20.882357 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.882323    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07"} err="failed to get container status \"c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07\": rpc error: code = NotFound desc = could not find container \"c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07\": container with ID starting with c61f17d986dc525858fa67df106262d92fa33eed86a869dca073464ef7189b07 not found: ID does not exist"
Jul 21 04:18:20.896168 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.896138    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d9f95f8cb-g9wcs"]
Jul 21 04:18:20.900228 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:20.900208    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d9f95f8cb-g9wcs"]
Jul 21 04:18:22.163403 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:22.163368    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf3d664-9507-43f7-a584-1585cc4690d9" path="/var/lib/kubelet/pods/fcf3d664-9507-43f7-a584-1585cc4690d9/volumes"
Jul 21 04:18:40.189604 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.189563    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:40.286661 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.286628    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:40.630783 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.630750    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:40.673457 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.673424    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"]
Jul 21 04:18:40.673819 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.673807    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf3d664-9507-43f7-a584-1585cc4690d9" containerName="manager"
Jul 21 04:18:40.673819 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.673821    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf3d664-9507-43f7-a584-1585cc4690d9" containerName="manager"
Jul 21 04:18:40.673944 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.673881    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf3d664-9507-43f7-a584-1585cc4690d9" containerName="manager"
Jul 21 04:18:40.678221 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.678198    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:40.680963 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.680940    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 21 04:18:40.681054 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.680983    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 04:18:40.681094 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.681047    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 04:18:40.688104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.688081    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"]
Jul 21 04:18:40.728263 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.728232    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0b0a511-8015-44ae-b26f-b1c100980544-maas-api-tls\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:40.728419 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.728293    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65t8t\" (UniqueName: \"kubernetes.io/projected/e0b0a511-8015-44ae-b26f-b1c100980544-kube-api-access-65t8t\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:40.728419 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.728359    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0b0a511-8015-44ae-b26f-b1c100980544-openshift-service-ca\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:40.829066 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.829028    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0b0a511-8015-44ae-b26f-b1c100980544-maas-api-tls\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:40.829261 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.829097    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65t8t\" (UniqueName: \"kubernetes.io/projected/e0b0a511-8015-44ae-b26f-b1c100980544-kube-api-access-65t8t\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:40.829261 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.829168    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0b0a511-8015-44ae-b26f-b1c100980544-openshift-service-ca\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:40.829261 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:18:40.829215    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 21 04:18:40.829378 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:18:40.829292    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0b0a511-8015-44ae-b26f-b1c100980544-maas-api-tls podName:e0b0a511-8015-44ae-b26f-b1c100980544 nodeName:}" failed. No retries permitted until 2026-07-21 04:18:41.329274564 +0000 UTC m=+541.714763463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e0b0a511-8015-44ae-b26f-b1c100980544-maas-api-tls") pod "maas-api-8666b4847-p2jw4" (UID: "e0b0a511-8015-44ae-b26f-b1c100980544") : secret "maas-api-serving-cert" not found
Jul 21 04:18:40.829803 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.829786    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0b0a511-8015-44ae-b26f-b1c100980544-openshift-service-ca\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:40.838599 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:40.838571    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65t8t\" (UniqueName: \"kubernetes.io/projected/e0b0a511-8015-44ae-b26f-b1c100980544-kube-api-access-65t8t\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:41.141294 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.141266    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:41.238642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.238609    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:41.246375 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.246343    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"]
Jul 21 04:18:41.249787 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.249769    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:18:41.252335 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.252315    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 21 04:18:41.252545 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.252527    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-x4rjw\""
Jul 21 04:18:41.259570 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.259549    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"]
Jul 21 04:18:41.264483 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.264462    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-4fdww"]
Jul 21 04:18:41.267692 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.267678    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:41.278636 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.278612    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-4fdww"]
Jul 21 04:18:41.333971 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.333946    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33c1fcd2-e914-4377-854c-7d2459113f21-plugins-config-volume\") pod \"payload-processing-5bf47cd988-4fdww\" (UID: \"33c1fcd2-e914-4377-854c-7d2459113f21\") " pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:41.334144 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.333989    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h25wj\" (UniqueName: \"kubernetes.io/projected/33c1fcd2-e914-4377-854c-7d2459113f21-kube-api-access-h25wj\") pod \"payload-processing-5bf47cd988-4fdww\" (UID: \"33c1fcd2-e914-4377-854c-7d2459113f21\") " pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:41.334144 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.334009    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4bzk\" (UniqueName: \"kubernetes.io/projected/df0c9777-01ce-4a56-bb96-d5431297daa3-kube-api-access-c4bzk\") pod \"payload-pre-processing-7677b774bb-bgjkd\" (UID: \"df0c9777-01ce-4a56-bb96-d5431297daa3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:18:41.334252 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.334143    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0b0a511-8015-44ae-b26f-b1c100980544-maas-api-tls\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:41.334252 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.334195    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df0c9777-01ce-4a56-bb96-d5431297daa3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-bgjkd\" (UID: \"df0c9777-01ce-4a56-bb96-d5431297daa3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:18:41.336601 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.336577    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0b0a511-8015-44ae-b26f-b1c100980544-maas-api-tls\") pod \"maas-api-8666b4847-p2jw4\" (UID: \"e0b0a511-8015-44ae-b26f-b1c100980544\") " pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:41.435331 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.435234    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df0c9777-01ce-4a56-bb96-d5431297daa3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-bgjkd\" (UID: \"df0c9777-01ce-4a56-bb96-d5431297daa3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:18:41.435331 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.435327    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33c1fcd2-e914-4377-854c-7d2459113f21-plugins-config-volume\") pod \"payload-processing-5bf47cd988-4fdww\" (UID: \"33c1fcd2-e914-4377-854c-7d2459113f21\") " pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:41.435610 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.435369    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h25wj\" (UniqueName: \"kubernetes.io/projected/33c1fcd2-e914-4377-854c-7d2459113f21-kube-api-access-h25wj\") pod \"payload-processing-5bf47cd988-4fdww\" (UID: \"33c1fcd2-e914-4377-854c-7d2459113f21\") " pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:41.435610 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.435385    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4bzk\" (UniqueName: \"kubernetes.io/projected/df0c9777-01ce-4a56-bb96-d5431297daa3-kube-api-access-c4bzk\") pod \"payload-pre-processing-7677b774bb-bgjkd\" (UID: \"df0c9777-01ce-4a56-bb96-d5431297daa3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:18:41.436046 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.436021    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df0c9777-01ce-4a56-bb96-d5431297daa3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-bgjkd\" (UID: \"df0c9777-01ce-4a56-bb96-d5431297daa3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:18:41.436156 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.436135    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33c1fcd2-e914-4377-854c-7d2459113f21-plugins-config-volume\") pod \"payload-processing-5bf47cd988-4fdww\" (UID: \"33c1fcd2-e914-4377-854c-7d2459113f21\") " pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:41.455333 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.455284    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4bzk\" (UniqueName: \"kubernetes.io/projected/df0c9777-01ce-4a56-bb96-d5431297daa3-kube-api-access-c4bzk\") pod \"payload-pre-processing-7677b774bb-bgjkd\" (UID: \"df0c9777-01ce-4a56-bb96-d5431297daa3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:18:41.455853 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.455834    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h25wj\" (UniqueName: \"kubernetes.io/projected/33c1fcd2-e914-4377-854c-7d2459113f21-kube-api-access-h25wj\") pod \"payload-processing-5bf47cd988-4fdww\" (UID: \"33c1fcd2-e914-4377-854c-7d2459113f21\") " pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:41.561291 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.561260    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:18:41.579640 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.579611    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:41.590031 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.590006    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:41.724958 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.724884    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"]
Jul 21 04:18:41.756795 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.756675    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-4fdww"]
Jul 21 04:18:41.759921 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:18:41.759894    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c1fcd2_e914_4377_854c_7d2459113f21.slice/crio-800e343280b0922b01399db6e91db998659c03ce6df5fd03278fd65fd4333b8e WatchSource:0}: Error finding container 800e343280b0922b01399db6e91db998659c03ce6df5fd03278fd65fd4333b8e: Status 404 returned error can't find the container with id 800e343280b0922b01399db6e91db998659c03ce6df5fd03278fd65fd4333b8e
Jul 21 04:18:41.780718 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.780633    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"]
Jul 21 04:18:41.782761 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:18:41.782736    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b0a511_8015_44ae_b26f_b1c100980544.slice/crio-cb8ea983a7b84a0a3fde37f3e9d12a33d387a3e2f0a57fcac016f90cf98c6fab WatchSource:0}: Error finding container cb8ea983a7b84a0a3fde37f3e9d12a33d387a3e2f0a57fcac016f90cf98c6fab: Status 404 returned error can't find the container with id cb8ea983a7b84a0a3fde37f3e9d12a33d387a3e2f0a57fcac016f90cf98c6fab
Jul 21 04:18:41.979470 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.979439    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4" event={"ID":"e0b0a511-8015-44ae-b26f-b1c100980544","Type":"ContainerStarted","Data":"cb8ea983a7b84a0a3fde37f3e9d12a33d387a3e2f0a57fcac016f90cf98c6fab"}
Jul 21 04:18:41.980671 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.980644    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd" event={"ID":"df0c9777-01ce-4a56-bb96-d5431297daa3","Type":"ContainerStarted","Data":"51cec7cdd1bbbefedbda4e31d30925a612a7bd0d2d6ae49e0e64920ab6aafaa3"}
Jul 21 04:18:41.981720 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:41.981701    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-4fdww" event={"ID":"33c1fcd2-e914-4377-854c-7d2459113f21","Type":"ContainerStarted","Data":"800e343280b0922b01399db6e91db998659c03ce6df5fd03278fd65fd4333b8e"}
Jul 21 04:18:43.995033 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:43.994989    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4" event={"ID":"e0b0a511-8015-44ae-b26f-b1c100980544","Type":"ContainerStarted","Data":"b8157bcd75c92fa5a5b756f16bc7b41830efeb49f6b50f0d2a60457e340cdbe2"}
Jul 21 04:18:43.995463 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:43.995089    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:43.996619 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:43.996593    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd" event={"ID":"df0c9777-01ce-4a56-bb96-d5431297daa3","Type":"ContainerStarted","Data":"3ccbc1f24f99bca835040f70d2657a07a7faf18ba5c5edeb33a4436da0058928"}
Jul 21 04:18:43.996741 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:43.996646    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:18:43.997892 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:43.997874    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-4fdww" event={"ID":"33c1fcd2-e914-4377-854c-7d2459113f21","Type":"ContainerStarted","Data":"0570a0fa097914ab3441e104fb26d76c68f63416e76bb88165c965d6bcb0eec8"}
Jul 21 04:18:43.997988 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:43.997920    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:44.013754 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:44.013711    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4" podStartSLOduration=1.9542900090000002 podStartE2EDuration="4.013697953s" totalImagesPullingTime="2.059407944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:18:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:18:44.012355142 +0000 UTC m=+544.397844052" watchObservedRunningTime="2026-07-21 04:18:44.013697953 +0000 UTC m=+544.399186862"
Jul 21 04:18:44.030130 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:44.030068    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd" podStartSLOduration=0.913779534 podStartE2EDuration="3.030046955s" totalImagesPullingTime="2.116267421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:18:44.027649024 +0000 UTC m=+544.413137934" watchObservedRunningTime="2026-07-21 04:18:44.030046955 +0000 UTC m=+544.415535865"
Jul 21 04:18:44.045528 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:44.045475    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-4fdww" podStartSLOduration=0.963983374 podStartE2EDuration="3.045461153s" totalImagesPullingTime="2.081477779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:18:44.043762624 +0000 UTC m=+544.429251533" watchObservedRunningTime="2026-07-21 04:18:44.045461153 +0000 UTC m=+544.430950062"
Jul 21 04:18:50.008503 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:50.008474    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8666b4847-p2jw4"
Jul 21 04:18:51.648475 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:51.648440    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:51.740788 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:51.740753    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:18:55.004959 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:55.004932    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-4fdww"
Jul 21 04:18:55.005441 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:18:55.004993    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-bgjkd"
Jul 21 04:19:04.118674 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:04.118637    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:19:18.271964 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:18.271933    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:19:28.881164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.881125    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"]
Jul 21 04:19:28.886473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.886453    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:28.893083 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.893053    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ftzmb\""
Jul 21 04:19:28.893224 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.893053    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 04:19:28.893224 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.893106    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 21 04:19:28.893224 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.893060    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 04:19:28.928280 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.928250    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"]
Jul 21 04:19:28.968302 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.968274    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:19:28.974860 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.974833    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:28.975008 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.974892    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:28.975008 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.974929    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj752\" (UniqueName: \"kubernetes.io/projected/2caf16b1-1d3d-443d-bcbf-835b39fda32a-kube-api-access-kj752\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:28.975164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.975064    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:28.975254 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.975103    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2caf16b1-1d3d-443d-bcbf-835b39fda32a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:28.975254 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:28.975224    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.076467 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.076434    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.076651 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.076539    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2caf16b1-1d3d-443d-bcbf-835b39fda32a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.076651 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.076588    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.076762 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.076663    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.076762 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.076709    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.076762 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.076740    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj752\" (UniqueName: \"kubernetes.io/projected/2caf16b1-1d3d-443d-bcbf-835b39fda32a-kube-api-access-kj752\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.076908 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.076857    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.076966 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.076934    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.077265 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.077235    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.079285 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.079257    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2caf16b1-1d3d-443d-bcbf-835b39fda32a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.079633 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.079613    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2caf16b1-1d3d-443d-bcbf-835b39fda32a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.087519 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.087490    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj752\" (UniqueName: \"kubernetes.io/projected/2caf16b1-1d3d-443d-bcbf-835b39fda32a-kube-api-access-kj752\") pod \"e2e-distinct-simulated-kserve-59975d69f9-94tlm\" (UID: \"2caf16b1-1d3d-443d-bcbf-835b39fda32a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.195852 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.195774    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:29.332275 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:29.332247    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"]
Jul 21 04:19:29.335685 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:19:29.335659    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2caf16b1_1d3d_443d_bcbf_835b39fda32a.slice/crio-5385804cc1501bd0bb17ed18a6b0d20111c8663f82b39b0b4f14b07c2eda014e WatchSource:0}: Error finding container 5385804cc1501bd0bb17ed18a6b0d20111c8663f82b39b0b4f14b07c2eda014e: Status 404 returned error can't find the container with id 5385804cc1501bd0bb17ed18a6b0d20111c8663f82b39b0b4f14b07c2eda014e
Jul 21 04:19:30.220933 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:30.220896    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm" event={"ID":"2caf16b1-1d3d-443d-bcbf-835b39fda32a","Type":"ContainerStarted","Data":"5385804cc1501bd0bb17ed18a6b0d20111c8663f82b39b0b4f14b07c2eda014e"}
Jul 21 04:19:31.227794 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:31.227759    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm" event={"ID":"2caf16b1-1d3d-443d-bcbf-835b39fda32a","Type":"ContainerStarted","Data":"18d345728183ee741848ff301e6dd98aacd9fcf0bab523624ed973a2fcf8bd2c"}
Jul 21 04:19:31.245972 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:31.245920    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm" podStartSLOduration=2.234425004 podStartE2EDuration="3.245903367s" totalImagesPullingTime="1.011478363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:19:31.245415614 +0000 UTC m=+591.630904524" watchObservedRunningTime="2026-07-21 04:19:31.245903367 +0000 UTC m=+591.631392277"
Jul 21 04:19:39.196841 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:39.196805    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:39.197318 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:39.196890    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:39.210009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:39.209986    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:39.283960 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:39.283935    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-94tlm"
Jul 21 04:19:40.043591 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:40.043565    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:19:40.043984 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:40.043964    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:19:45.114522 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:45.114488    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:19:54.992302 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:19:54.992260    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:20:10.512447 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:20:10.512405    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:20:20.734416 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:20:20.734380    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:20:20.793841 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:20:20.793807    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:20:20.911363 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:20:20.911305    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:21:16.358596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:21:16.358557    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:21:17.657566 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:21:17.657525    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:21:17.754276 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:21:17.754243    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:21:18.858326 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:21:18.858292    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:21:18.956930 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:21:18.956893    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:21:37.629250 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:21:37.629215    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:22:04.550355 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.550318    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-rmstp"]
Jul 21 04:22:04.550882 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.550537    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp" podUID="ce1aa7c8-813e-4340-9486-ed68001ff6e6" containerName="manager" containerID="cri-o://5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305" gracePeriod=30
Jul 21 04:22:04.805403 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.805334    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:22:04.905651 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.905616    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hw89\" (UniqueName: \"kubernetes.io/projected/ce1aa7c8-813e-4340-9486-ed68001ff6e6-kube-api-access-2hw89\") pod \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\" (UID: \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\") "
Jul 21 04:22:04.905820 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.905730    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce1aa7c8-813e-4340-9486-ed68001ff6e6-webhook-cert\") pod \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\" (UID: \"ce1aa7c8-813e-4340-9486-ed68001ff6e6\") "
Jul 21 04:22:04.907765 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.907737    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce1aa7c8-813e-4340-9486-ed68001ff6e6-kube-api-access-2hw89" (OuterVolumeSpecName: "kube-api-access-2hw89") pod "ce1aa7c8-813e-4340-9486-ed68001ff6e6" (UID: "ce1aa7c8-813e-4340-9486-ed68001ff6e6"). InnerVolumeSpecName "kube-api-access-2hw89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:22:04.907866 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.907784    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce1aa7c8-813e-4340-9486-ed68001ff6e6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ce1aa7c8-813e-4340-9486-ed68001ff6e6" (UID: "ce1aa7c8-813e-4340-9486-ed68001ff6e6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:22:04.989734 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.989699    2546 generic.go:356] "Generic (PLEG): container finished" podID="ce1aa7c8-813e-4340-9486-ed68001ff6e6" containerID="5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305" exitCode=0
Jul 21 04:22:04.989899 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.989746    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp"
Jul 21 04:22:04.989899 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.989751    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp" event={"ID":"ce1aa7c8-813e-4340-9486-ed68001ff6e6","Type":"ContainerDied","Data":"5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305"}
Jul 21 04:22:04.989899 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.989798    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-rmstp" event={"ID":"ce1aa7c8-813e-4340-9486-ed68001ff6e6","Type":"ContainerDied","Data":"1e87aec6c9e89b6710f49fd05ea761164a675e2c875672433318b949043021d6"}
Jul 21 04:22:04.989899 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.989820    2546 scope.go:117] "RemoveContainer" containerID="5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305"
Jul 21 04:22:04.998682 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.998665    2546 scope.go:117] "RemoveContainer" containerID="5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305"
Jul 21 04:22:04.998944 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:22:04.998925    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305\": container with ID starting with 5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305 not found: ID does not exist" containerID="5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305"
Jul 21 04:22:04.998992 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:04.998953    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305"} err="failed to get container status \"5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305\": rpc error: code = NotFound desc = could not find container \"5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305\": container with ID starting with 5a3dada8096beefbf9674582f0bd86a8edde292ec685ef59016bcdd427cf6305 not found: ID does not exist"
Jul 21 04:22:05.007057 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:05.007036    2546 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce1aa7c8-813e-4340-9486-ed68001ff6e6-webhook-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:22:05.007184 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:05.007061    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hw89\" (UniqueName: \"kubernetes.io/projected/ce1aa7c8-813e-4340-9486-ed68001ff6e6-kube-api-access-2hw89\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:22:05.012563 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:05.012540    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-rmstp"]
Jul 21 04:22:05.016208 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:05.016188    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-rmstp"]
Jul 21 04:22:06.163044 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:22:06.163012    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce1aa7c8-813e-4340-9486-ed68001ff6e6" path="/var/lib/kubelet/pods/ce1aa7c8-813e-4340-9486-ed68001ff6e6/volumes"
Jul 21 04:23:05.834458 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.834419    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-kprxf"]
Jul 21 04:23:05.834964 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.834920    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce1aa7c8-813e-4340-9486-ed68001ff6e6" containerName="manager"
Jul 21 04:23:05.834964 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.834938    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce1aa7c8-813e-4340-9486-ed68001ff6e6" containerName="manager"
Jul 21 04:23:05.835071 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.835051    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce1aa7c8-813e-4340-9486-ed68001ff6e6" containerName="manager"
Jul 21 04:23:05.838382 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.838360    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:05.842173 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.842155    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 04:23:05.842173 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.842163    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 04:23:05.842328 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.842159    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4nl9r\""
Jul 21 04:23:05.847711 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.847687    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-kprxf"]
Jul 21 04:23:05.963598 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.963566    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkntw\" (UniqueName: \"kubernetes.io/projected/98a12538-1697-4873-8d90-d469487c1079-kube-api-access-nkntw\") pod \"maas-controller-6ffdf96fc7-kprxf\" (UID: \"98a12538-1697-4873-8d90-d469487c1079\") " pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:05.963776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:05.963634    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98a12538-1697-4873-8d90-d469487c1079-webhook-cert\") pod \"maas-controller-6ffdf96fc7-kprxf\" (UID: \"98a12538-1697-4873-8d90-d469487c1079\") " pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:06.065073 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:06.065037    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkntw\" (UniqueName: \"kubernetes.io/projected/98a12538-1697-4873-8d90-d469487c1079-kube-api-access-nkntw\") pod \"maas-controller-6ffdf96fc7-kprxf\" (UID: \"98a12538-1697-4873-8d90-d469487c1079\") " pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:06.065246 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:06.065125    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98a12538-1697-4873-8d90-d469487c1079-webhook-cert\") pod \"maas-controller-6ffdf96fc7-kprxf\" (UID: \"98a12538-1697-4873-8d90-d469487c1079\") " pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:06.067620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:06.067602    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98a12538-1697-4873-8d90-d469487c1079-webhook-cert\") pod \"maas-controller-6ffdf96fc7-kprxf\" (UID: \"98a12538-1697-4873-8d90-d469487c1079\") " pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:06.074672 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:06.074645    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkntw\" (UniqueName: \"kubernetes.io/projected/98a12538-1697-4873-8d90-d469487c1079-kube-api-access-nkntw\") pod \"maas-controller-6ffdf96fc7-kprxf\" (UID: \"98a12538-1697-4873-8d90-d469487c1079\") " pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:06.150916 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:06.150828    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:06.289029 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:06.289006    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-kprxf"]
Jul 21 04:23:06.291450 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:23:06.291424    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a12538_1697_4873_8d90_d469487c1079.slice/crio-df57136aadc1fd31c471dab65b80c3e698a5d68916e3d5856038504801d93f01 WatchSource:0}: Error finding container df57136aadc1fd31c471dab65b80c3e698a5d68916e3d5856038504801d93f01: Status 404 returned error can't find the container with id df57136aadc1fd31c471dab65b80c3e698a5d68916e3d5856038504801d93f01
Jul 21 04:23:06.293109 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:06.293089    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 04:23:07.289886 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:07.289852    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf" event={"ID":"98a12538-1697-4873-8d90-d469487c1079","Type":"ContainerStarted","Data":"82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce"}
Jul 21 04:23:07.290356 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:07.289911    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf" event={"ID":"98a12538-1697-4873-8d90-d469487c1079","Type":"ContainerStarted","Data":"df57136aadc1fd31c471dab65b80c3e698a5d68916e3d5856038504801d93f01"}
Jul 21 04:23:07.290356 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:07.290039    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:07.310392 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:07.310343    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf" podStartSLOduration=1.6999938559999999 podStartE2EDuration="2.310327388s" totalImagesPullingTime="610.333532ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:23:07.308211415 +0000 UTC m=+807.693700337" watchObservedRunningTime="2026-07-21 04:23:07.310327388 +0000 UTC m=+807.695816332"
Jul 21 04:23:18.296982 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:18.296946    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:21.973277 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:21.973243    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:23:22.839005 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:22.838974    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:23:22.946133 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:22.946079    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-kprxf"]
Jul 21 04:23:22.946298 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:22.946277    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf" podUID="98a12538-1697-4873-8d90-d469487c1079" containerName="manager" containerID="cri-o://82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce" gracePeriod=30
Jul 21 04:23:23.200629 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.200606    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:23.321787 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.321748    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkntw\" (UniqueName: \"kubernetes.io/projected/98a12538-1697-4873-8d90-d469487c1079-kube-api-access-nkntw\") pod \"98a12538-1697-4873-8d90-d469487c1079\" (UID: \"98a12538-1697-4873-8d90-d469487c1079\") "
Jul 21 04:23:23.321982 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.321835    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98a12538-1697-4873-8d90-d469487c1079-webhook-cert\") pod \"98a12538-1697-4873-8d90-d469487c1079\" (UID: \"98a12538-1697-4873-8d90-d469487c1079\") "
Jul 21 04:23:23.324058 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.324030    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98a12538-1697-4873-8d90-d469487c1079-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "98a12538-1697-4873-8d90-d469487c1079" (UID: "98a12538-1697-4873-8d90-d469487c1079"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:23:23.324199 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.324167    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a12538-1697-4873-8d90-d469487c1079-kube-api-access-nkntw" (OuterVolumeSpecName: "kube-api-access-nkntw") pod "98a12538-1697-4873-8d90-d469487c1079" (UID: "98a12538-1697-4873-8d90-d469487c1079"). InnerVolumeSpecName "kube-api-access-nkntw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:23:23.370503 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.370431    2546 generic.go:356] "Generic (PLEG): container finished" podID="98a12538-1697-4873-8d90-d469487c1079" containerID="82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce" exitCode=0
Jul 21 04:23:23.370503 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.370483    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf"
Jul 21 04:23:23.370503 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.370493    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf" event={"ID":"98a12538-1697-4873-8d90-d469487c1079","Type":"ContainerDied","Data":"82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce"}
Jul 21 04:23:23.370714 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.370559    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-kprxf" event={"ID":"98a12538-1697-4873-8d90-d469487c1079","Type":"ContainerDied","Data":"df57136aadc1fd31c471dab65b80c3e698a5d68916e3d5856038504801d93f01"}
Jul 21 04:23:23.370714 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.370581    2546 scope.go:117] "RemoveContainer" containerID="82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce"
Jul 21 04:23:23.379305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.379285    2546 scope.go:117] "RemoveContainer" containerID="82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce"
Jul 21 04:23:23.379553 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:23:23.379535    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce\": container with ID starting with 82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce not found: ID does not exist" containerID="82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce"
Jul 21 04:23:23.379610 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.379561    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce"} err="failed to get container status \"82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce\": rpc error: code = NotFound desc = could not find container \"82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce\": container with ID starting with 82bb0323968ca8ea68658579cf069f10636d00426c021899d187cc2d806274ce not found: ID does not exist"
Jul 21 04:23:23.393691 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.393669    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-kprxf"]
Jul 21 04:23:23.397658 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.397638    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-kprxf"]
Jul 21 04:23:23.423341 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.423319    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkntw\" (UniqueName: \"kubernetes.io/projected/98a12538-1697-4873-8d90-d469487c1079-kube-api-access-nkntw\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:23:23.423341 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:23.423340    2546 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98a12538-1697-4873-8d90-d469487c1079-webhook-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:23:24.039838 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.039801    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:23:24.163289 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.163242    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98a12538-1697-4873-8d90-d469487c1079" path="/var/lib/kubelet/pods/98a12538-1697-4873-8d90-d469487c1079/volumes"
Jul 21 04:23:24.163591 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.163574    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-dknzk"]
Jul 21 04:23:24.164004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.163991    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a12538-1697-4873-8d90-d469487c1079" containerName="manager"
Jul 21 04:23:24.164045 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.164006    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a12538-1697-4873-8d90-d469487c1079" containerName="manager"
Jul 21 04:23:24.164092 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.164083    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a12538-1697-4873-8d90-d469487c1079" containerName="manager"
Jul 21 04:23:24.168498 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.168478    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:23:24.171595 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.171576    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 04:23:24.171709 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.171613    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 04:23:24.178909 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.178888    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4nl9r\""
Jul 21 04:23:24.179258 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.179230    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-dknzk"]
Jul 21 04:23:24.229021 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.228993    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjvzj\" (UniqueName: \"kubernetes.io/projected/b07b3a5d-dde8-462d-b3ed-02f20b685664-kube-api-access-xjvzj\") pod \"maas-controller-6ffdf96fc7-dknzk\" (UID: \"b07b3a5d-dde8-462d-b3ed-02f20b685664\") " pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:23:24.229396 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.229063    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b07b3a5d-dde8-462d-b3ed-02f20b685664-webhook-cert\") pod \"maas-controller-6ffdf96fc7-dknzk\" (UID: \"b07b3a5d-dde8-462d-b3ed-02f20b685664\") " pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:23:24.329878 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.329803    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b07b3a5d-dde8-462d-b3ed-02f20b685664-webhook-cert\") pod \"maas-controller-6ffdf96fc7-dknzk\" (UID: \"b07b3a5d-dde8-462d-b3ed-02f20b685664\") " pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:23:24.329878 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.329869    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjvzj\" (UniqueName: \"kubernetes.io/projected/b07b3a5d-dde8-462d-b3ed-02f20b685664-kube-api-access-xjvzj\") pod \"maas-controller-6ffdf96fc7-dknzk\" (UID: \"b07b3a5d-dde8-462d-b3ed-02f20b685664\") " pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:23:24.332310 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.332291    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b07b3a5d-dde8-462d-b3ed-02f20b685664-webhook-cert\") pod \"maas-controller-6ffdf96fc7-dknzk\" (UID: \"b07b3a5d-dde8-462d-b3ed-02f20b685664\") " pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:23:24.340011 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.339982    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjvzj\" (UniqueName: \"kubernetes.io/projected/b07b3a5d-dde8-462d-b3ed-02f20b685664-kube-api-access-xjvzj\") pod \"maas-controller-6ffdf96fc7-dknzk\" (UID: \"b07b3a5d-dde8-462d-b3ed-02f20b685664\") " pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:23:24.480501 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.480471    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:23:24.616440 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:24.616416    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6ffdf96fc7-dknzk"]
Jul 21 04:23:24.618641 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:23:24.618614    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07b3a5d_dde8_462d_b3ed_02f20b685664.slice/crio-f4882dcbf49f4e35d4ced7a918539e0d0f34d7c72f7e91b7391c52ecff6ad329 WatchSource:0}: Error finding container f4882dcbf49f4e35d4ced7a918539e0d0f34d7c72f7e91b7391c52ecff6ad329: Status 404 returned error can't find the container with id f4882dcbf49f4e35d4ced7a918539e0d0f34d7c72f7e91b7391c52ecff6ad329
Jul 21 04:23:25.383100 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:25.383062    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-dknzk" event={"ID":"b07b3a5d-dde8-462d-b3ed-02f20b685664","Type":"ContainerStarted","Data":"9c5adc3bca4eebe99cf1260045e939395f0e0939d168271133de64fce8167097"}
Jul 21 04:23:25.383501 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:25.383147    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ffdf96fc7-dknzk" event={"ID":"b07b3a5d-dde8-462d-b3ed-02f20b685664","Type":"ContainerStarted","Data":"f4882dcbf49f4e35d4ced7a918539e0d0f34d7c72f7e91b7391c52ecff6ad329"}
Jul 21 04:23:25.383501 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:25.383182    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:23:25.405071 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:25.405021    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6ffdf96fc7-dknzk" podStartSLOduration=1.059110987 podStartE2EDuration="1.405004589s" totalImagesPullingTime="345.893602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:23:25.402583194 +0000 UTC m=+825.788072130" watchObservedRunningTime="2026-07-21 04:23:25.405004589 +0000 UTC m=+825.790493514"
Jul 21 04:23:36.392499 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:23:36.392468    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6ffdf96fc7-dknzk"
Jul 21 04:24:04.669501 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:04.669459    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:04.762524 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:04.762487    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:04.876592 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:04.876559    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:10.861390 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:10.861355    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:10.921332 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:10.921301    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:11.004696 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:11.004662    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:11.104897 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:11.104868    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:11.217744 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:11.217714    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:17.103595 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:17.103566    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:17.203666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:17.203627    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:22.576583 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:22.576549    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:22.678625 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:22.678593    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:27.145469 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:27.145420    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:24:40.073632 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:40.073599    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:24:40.075831 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:24:40.075805    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:27:46.792014 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:27:46.791932    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:27:48.501043 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:27:48.501010    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:28:42.320165 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:28:42.320105    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:28:42.388352 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:28:42.388317    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:29:40.107726 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:29:40.107701    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:29:40.111156 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:29:40.111132    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:30:00.154974 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:00.154940    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25"]
Jul 21 04:30:00.159260 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:00.159240    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25"
Jul 21 04:30:00.162160 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:00.162138    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:30:00.167760 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:00.167731    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25"]
Jul 21 04:30:00.173470 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:00.173442    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25"
Jul 21 04:30:00.312769 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:00.312745    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25"]
Jul 21 04:30:00.315150 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:30:00.315101    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6548d8_08a4_4988_9046_fad547dbffbb.slice/crio-52b81c4cd6e3542404b043195177e80b345318ef745050282a49d994d5de1687 WatchSource:0}: Error finding container 52b81c4cd6e3542404b043195177e80b345318ef745050282a49d994d5de1687: Status 404 returned error can't find the container with id 52b81c4cd6e3542404b043195177e80b345318ef745050282a49d994d5de1687
Jul 21 04:30:00.317045 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:00.317027    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 04:30:00.324398 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:00.324374    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25" event={"ID":"ee6548d8-08a4-4988-9046-fad547dbffbb","Type":"ContainerStarted","Data":"52b81c4cd6e3542404b043195177e80b345318ef745050282a49d994d5de1687"}
Jul 21 04:30:02.716074 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:02.716036    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:30:23.436407 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:23.436372    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25" event={"ID":"ee6548d8-08a4-4988-9046-fad547dbffbb","Type":"ContainerStarted","Data":"b5d1d8b6baa9fe146ada67a8700365119b8bdf850fe91aee512f826aad4ff96c"}
Jul 21 04:30:23.454580 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:23.454527    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25" podStartSLOduration=0.766295143 podStartE2EDuration="23.454512901s" totalImagesPullingTime="22.688217758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:30:23.453085788 +0000 UTC m=+1243.838574723" watchObservedRunningTime="2026-07-21 04:30:23.454512901 +0000 UTC m=+1243.840001833"
Jul 21 04:30:24.442571 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:24.442538    2546 generic.go:356] "Generic (PLEG): container finished" podID="ee6548d8-08a4-4988-9046-fad547dbffbb" containerID="b5d1d8b6baa9fe146ada67a8700365119b8bdf850fe91aee512f826aad4ff96c" exitCode=0
Jul 21 04:30:24.442981 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:24.442599    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25" event={"ID":"ee6548d8-08a4-4988-9046-fad547dbffbb","Type":"ContainerDied","Data":"b5d1d8b6baa9fe146ada67a8700365119b8bdf850fe91aee512f826aad4ff96c"}
Jul 21 04:30:25.583946 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:25.583922    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25"
Jul 21 04:30:26.454650 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:26.454571    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25"
Jul 21 04:30:26.454650 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:26.454606    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743470-xjk25" event={"ID":"ee6548d8-08a4-4988-9046-fad547dbffbb","Type":"ContainerDied","Data":"52b81c4cd6e3542404b043195177e80b345318ef745050282a49d994d5de1687"}
Jul 21 04:30:26.454838 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:30:26.454654    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52b81c4cd6e3542404b043195177e80b345318ef745050282a49d994d5de1687"
Jul 21 04:31:59.446263 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:31:59.446228    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:32:15.331751 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:32:15.331704    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:32:37.594678 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:32:37.594626    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:32:38.534694 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:32:38.534655    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:32:51.505903 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:32:51.505865    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:33:23.051962 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:33:23.051922    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:33:23.742624 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:33:23.742585    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:34:40.138873 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:34:40.138846    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:34:40.150631 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:34:40.150602    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:35:43.916127 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:35:43.916086    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:35:51.568992 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:35:51.568955    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:35:59.872526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:35:59.872490    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:06.296645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:06.296603    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:09.504063 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:09.504031    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:09.603702 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:09.603657    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:09.972446 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:09.972411    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:16.138131 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:16.138085    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:16.202014 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:16.201985    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:16.374414 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:16.374377    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:16.694844 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:16.694814    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:16.983822 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:16.983790    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:19.478978 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.478940    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:19.542455 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.542411    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"]
Jul 21 04:36:19.543021 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.542996    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee6548d8-08a4-4988-9046-fad547dbffbb" containerName="cleanup"
Jul 21 04:36:19.543021 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.543020    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee6548d8-08a4-4988-9046-fad547dbffbb" containerName="cleanup"
Jul 21 04:36:19.543220 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.543138    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee6548d8-08a4-4988-9046-fad547dbffbb" containerName="cleanup"
Jul 21 04:36:19.546175 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.546161    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.548508 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.548483    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fc7e6220-serving-cert\""
Jul 21 04:36:19.548645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.548493    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gvc8n\""
Jul 21 04:36:19.559521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.559489    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"]
Jul 21 04:36:19.562547 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.562522    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:19.592629 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.592576    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a873af7-1d31-40d8-a928-56e9611b25c2-openshift-service-ca\") pod \"maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.592806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.592728    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4td2\" (UniqueName: \"kubernetes.io/projected/3a873af7-1d31-40d8-a928-56e9611b25c2-kube-api-access-s4td2\") pod \"maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.592806 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.592770    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a873af7-1d31-40d8-a928-56e9611b25c2-maas-api-tls\") pod \"maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.628421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.628393    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:19.693816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.693783    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4td2\" (UniqueName: \"kubernetes.io/projected/3a873af7-1d31-40d8-a928-56e9611b25c2-kube-api-access-s4td2\") pod \"maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.694015 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.693839    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a873af7-1d31-40d8-a928-56e9611b25c2-maas-api-tls\") pod \"maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.694015 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.693903    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a873af7-1d31-40d8-a928-56e9611b25c2-openshift-service-ca\") pod \"maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.695055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.694981    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a873af7-1d31-40d8-a928-56e9611b25c2-openshift-service-ca\") pod \"maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.697334 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.697312    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a873af7-1d31-40d8-a928-56e9611b25c2-maas-api-tls\") pod \"maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.704301 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.704272    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4td2\" (UniqueName: \"kubernetes.io/projected/3a873af7-1d31-40d8-a928-56e9611b25c2-kube-api-access-s4td2\") pod \"maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:19.727137 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.727083    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:19.857808 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:19.857725    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:20.041067 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:20.040994    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:20.149467 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:20.149380    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:20.216137 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:20.216086    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"]
Jul 21 04:36:20.222174 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:36:20.222147    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a873af7_1d31_40d8_a928_56e9611b25c2.slice/crio-4c85e9d41cce1b896dcbe7296fb7031247dbacb5073cc0194f440b4e7b71da10 WatchSource:0}: Error finding container 4c85e9d41cce1b896dcbe7296fb7031247dbacb5073cc0194f440b4e7b71da10: Status 404 returned error can't find the container with id 4c85e9d41cce1b896dcbe7296fb7031247dbacb5073cc0194f440b4e7b71da10
Jul 21 04:36:20.223671 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:20.223650    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 04:36:21.224096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:21.224057    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln" event={"ID":"3a873af7-1d31-40d8-a928-56e9611b25c2","Type":"ContainerStarted","Data":"8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5"}
Jul 21 04:36:21.224540 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:21.224184    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:21.224540 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:21.224210    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln" event={"ID":"3a873af7-1d31-40d8-a928-56e9611b25c2","Type":"ContainerStarted","Data":"4c85e9d41cce1b896dcbe7296fb7031247dbacb5073cc0194f440b4e7b71da10"}
Jul 21 04:36:21.243680 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:21.243632    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln" podStartSLOduration=1.772923299 podStartE2EDuration="2.243616869s" totalImagesPullingTime="470.69357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:36:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:36:21.241686817 +0000 UTC m=+1601.627175727" watchObservedRunningTime="2026-07-21 04:36:21.243616869 +0000 UTC m=+1601.629105778"
Jul 21 04:36:27.234751 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:27.234723    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:45.152991 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:45.152894    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"]
Jul 21 04:36:45.157674 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:45.157646    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"
Jul 21 04:36:45.160088 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:45.160067    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:36:45.163968 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:45.163946    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"]
Jul 21 04:36:45.330450 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:45.330412    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96720e9f-86a6-440e-a7d1-38e0dfef93f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5\" (UID: \"96720e9f-86a6-440e-a7d1-38e0dfef93f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"
Jul 21 04:36:45.432033 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:45.431959    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96720e9f-86a6-440e-a7d1-38e0dfef93f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5\" (UID: \"96720e9f-86a6-440e-a7d1-38e0dfef93f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"
Jul 21 04:36:45.432546 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:45.432530    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96720e9f-86a6-440e-a7d1-38e0dfef93f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5\" (UID: \"96720e9f-86a6-440e-a7d1-38e0dfef93f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"
Jul 21 04:36:45.470761 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:45.470734    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"
Jul 21 04:36:45.610064 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:45.610039    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"]
Jul 21 04:36:45.612188 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:36:45.612156    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96720e9f_86a6_440e_a7d1_38e0dfef93f3.slice/crio-6ddba7dd6f4536ee1704b84e53e57acff8bf78a9cd0e32e2b4e9472bf8479dd2 WatchSource:0}: Error finding container 6ddba7dd6f4536ee1704b84e53e57acff8bf78a9cd0e32e2b4e9472bf8479dd2: Status 404 returned error can't find the container with id 6ddba7dd6f4536ee1704b84e53e57acff8bf78a9cd0e32e2b4e9472bf8479dd2
Jul 21 04:36:46.353257 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:46.353230    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5" event={"ID":"96720e9f-86a6-440e-a7d1-38e0dfef93f3","Type":"ContainerStarted","Data":"1293e63dd7a34e31bd2186e6a25cc4b9b28d14959ebc4683d5674a59898e737d"}
Jul 21 04:36:46.353593 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:46.353291    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5" event={"ID":"96720e9f-86a6-440e-a7d1-38e0dfef93f3","Type":"ContainerStarted","Data":"6ddba7dd6f4536ee1704b84e53e57acff8bf78a9cd0e32e2b4e9472bf8479dd2"}
Jul 21 04:36:47.359779 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:47.359744    2546 generic.go:356] "Generic (PLEG): container finished" podID="96720e9f-86a6-440e-a7d1-38e0dfef93f3" containerID="1293e63dd7a34e31bd2186e6a25cc4b9b28d14959ebc4683d5674a59898e737d" exitCode=0
Jul 21 04:36:47.360396 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:47.359823    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5" event={"ID":"96720e9f-86a6-440e-a7d1-38e0dfef93f3","Type":"ContainerDied","Data":"1293e63dd7a34e31bd2186e6a25cc4b9b28d14959ebc4683d5674a59898e737d"}
Jul 21 04:36:47.501189 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:47.501166    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"
Jul 21 04:36:47.655785 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:47.655699    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96720e9f-86a6-440e-a7d1-38e0dfef93f3-maas-api-service-ca\") pod \"96720e9f-86a6-440e-a7d1-38e0dfef93f3\" (UID: \"96720e9f-86a6-440e-a7d1-38e0dfef93f3\") "
Jul 21 04:36:47.656055 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:47.656033    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96720e9f-86a6-440e-a7d1-38e0dfef93f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96720e9f-86a6-440e-a7d1-38e0dfef93f3" (UID: "96720e9f-86a6-440e-a7d1-38e0dfef93f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:36:47.756838 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:47.756808    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96720e9f-86a6-440e-a7d1-38e0dfef93f3-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:36:48.365472 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:48.365442    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"
Jul 21 04:36:48.365472 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:48.365465    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5" event={"ID":"96720e9f-86a6-440e-a7d1-38e0dfef93f3","Type":"ContainerDied","Data":"6ddba7dd6f4536ee1704b84e53e57acff8bf78a9cd0e32e2b4e9472bf8479dd2"}
Jul 21 04:36:48.365877 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:48.365535    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ddba7dd6f4536ee1704b84e53e57acff8bf78a9cd0e32e2b4e9472bf8479dd2"
Jul 21 04:36:55.668360 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:55.668329    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"]
Jul 21 04:36:55.668824 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:55.668590    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln" podUID="3a873af7-1d31-40d8-a928-56e9611b25c2" containerName="maas-api" containerID="cri-o://8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5" gracePeriod=30
Jul 21 04:36:55.692244 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:55.692220    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:36:55.925431 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:55.925371    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:56.033216 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.033181    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4td2\" (UniqueName: \"kubernetes.io/projected/3a873af7-1d31-40d8-a928-56e9611b25c2-kube-api-access-s4td2\") pod \"3a873af7-1d31-40d8-a928-56e9611b25c2\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") "
Jul 21 04:36:56.033403 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.033282    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a873af7-1d31-40d8-a928-56e9611b25c2-maas-api-tls\") pod \"3a873af7-1d31-40d8-a928-56e9611b25c2\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") "
Jul 21 04:36:56.033403 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.033328    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a873af7-1d31-40d8-a928-56e9611b25c2-openshift-service-ca\") pod \"3a873af7-1d31-40d8-a928-56e9611b25c2\" (UID: \"3a873af7-1d31-40d8-a928-56e9611b25c2\") "
Jul 21 04:36:56.033717 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.033692    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a873af7-1d31-40d8-a928-56e9611b25c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3a873af7-1d31-40d8-a928-56e9611b25c2" (UID: "3a873af7-1d31-40d8-a928-56e9611b25c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:36:56.035462 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.035438    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a873af7-1d31-40d8-a928-56e9611b25c2-kube-api-access-s4td2" (OuterVolumeSpecName: "kube-api-access-s4td2") pod "3a873af7-1d31-40d8-a928-56e9611b25c2" (UID: "3a873af7-1d31-40d8-a928-56e9611b25c2"). InnerVolumeSpecName "kube-api-access-s4td2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:36:56.035566 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.035546    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a873af7-1d31-40d8-a928-56e9611b25c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3a873af7-1d31-40d8-a928-56e9611b25c2" (UID: "3a873af7-1d31-40d8-a928-56e9611b25c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:36:56.134456 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.134425    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a873af7-1d31-40d8-a928-56e9611b25c2-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:36:56.134456 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.134453    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a873af7-1d31-40d8-a928-56e9611b25c2-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:36:56.134456 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.134462    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4td2\" (UniqueName: \"kubernetes.io/projected/3a873af7-1d31-40d8-a928-56e9611b25c2-kube-api-access-s4td2\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:36:56.407338 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.407303    2546 generic.go:356] "Generic (PLEG): container finished" podID="3a873af7-1d31-40d8-a928-56e9611b25c2" containerID="8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5" exitCode=0
Jul 21 04:36:56.407559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.407348    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"
Jul 21 04:36:56.407559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.407389    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln" event={"ID":"3a873af7-1d31-40d8-a928-56e9611b25c2","Type":"ContainerDied","Data":"8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5"}
Jul 21 04:36:56.407559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.407452    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln" event={"ID":"3a873af7-1d31-40d8-a928-56e9611b25c2","Type":"ContainerDied","Data":"4c85e9d41cce1b896dcbe7296fb7031247dbacb5073cc0194f440b4e7b71da10"}
Jul 21 04:36:56.407559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.407484    2546 scope.go:117] "RemoveContainer" containerID="8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5"
Jul 21 04:36:56.416317 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.416300    2546 scope.go:117] "RemoveContainer" containerID="8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5"
Jul 21 04:36:56.416572 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:36:56.416555    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5\": container with ID starting with 8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5 not found: ID does not exist" containerID="8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5"
Jul 21 04:36:56.416619 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.416585    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5"} err="failed to get container status \"8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5\": rpc error: code = NotFound desc = could not find container \"8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5\": container with ID starting with 8bd61c221fdb78ca73986d8971af5e39807462d4b671b11264dad5ad609fd5f5 not found: ID does not exist"
Jul 21 04:36:56.425898 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.425876    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"]
Jul 21 04:36:56.430626 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:56.430607    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc7e6220-66fb5564d9-wc8ln"]
Jul 21 04:36:58.163707 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:36:58.163671    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a873af7-1d31-40d8-a928-56e9611b25c2" path="/var/lib/kubelet/pods/3a873af7-1d31-40d8-a928-56e9611b25c2/volumes"
Jul 21 04:37:00.228660 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:00.228623    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:00.231052 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:00.229584    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"]
Jul 21 04:37:00.232699 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:00.232681    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc7e6220-zsjj5"]
Jul 21 04:37:00.397271 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:00.397243    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:00.638675 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:00.638595    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:00.907715 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:00.907630    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:01.000424 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:01.000393    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:01.125299 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:01.125264    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:01.437150 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:01.437088    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:01.794091 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:01.794057    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:02.163044 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:02.162964    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96720e9f-86a6-440e-a7d1-38e0dfef93f3" path="/var/lib/kubelet/pods/96720e9f-86a6-440e-a7d1-38e0dfef93f3/volumes"
Jul 21 04:37:03.872608 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.872580    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:03.943665 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.943634    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"]
Jul 21 04:37:03.944108 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.944093    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a873af7-1d31-40d8-a928-56e9611b25c2" containerName="maas-api"
Jul 21 04:37:03.944181 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.944126    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a873af7-1d31-40d8-a928-56e9611b25c2" containerName="maas-api"
Jul 21 04:37:03.944181 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.944149    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96720e9f-86a6-440e-a7d1-38e0dfef93f3" containerName="revoke-keys"
Jul 21 04:37:03.944181 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.944155    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="96720e9f-86a6-440e-a7d1-38e0dfef93f3" containerName="revoke-keys"
Jul 21 04:37:03.944293 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.944224    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a873af7-1d31-40d8-a928-56e9611b25c2" containerName="maas-api"
Jul 21 04:37:03.944293 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.944233    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="96720e9f-86a6-440e-a7d1-38e0dfef93f3" containerName="revoke-keys"
Jul 21 04:37:03.947072 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.947045    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:03.949358 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.949335    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c0006c3d-serving-cert\""
Jul 21 04:37:03.949696 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.949677    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gvc8n\""
Jul 21 04:37:03.956291 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.956270    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"]
Jul 21 04:37:03.974210 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:03.974178    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:04.004556 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.004526    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftkgb\" (UniqueName: \"kubernetes.io/projected/85efe203-177e-408c-94e6-abd885b99d55-kube-api-access-ftkgb\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.004693 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.004568    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85efe203-177e-408c-94e6-abd885b99d55-openshift-service-ca\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.004693 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.004628    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85efe203-177e-408c-94e6-abd885b99d55-maas-api-tls\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.105192 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.105152    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftkgb\" (UniqueName: \"kubernetes.io/projected/85efe203-177e-408c-94e6-abd885b99d55-kube-api-access-ftkgb\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.105385 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.105242    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85efe203-177e-408c-94e6-abd885b99d55-openshift-service-ca\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.105385 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.105306    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85efe203-177e-408c-94e6-abd885b99d55-maas-api-tls\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.105525 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:37:04.105445    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-serving-cert: secret "maas-api-e2e-migrate-c0006c3d-serving-cert" not found
Jul 21 04:37:04.105525 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:37:04.105498    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85efe203-177e-408c-94e6-abd885b99d55-maas-api-tls podName:85efe203-177e-408c-94e6-abd885b99d55 nodeName:}" failed. No retries permitted until 2026-07-21 04:37:04.605482911 +0000 UTC m=+1644.990971799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/85efe203-177e-408c-94e6-abd885b99d55-maas-api-tls") pod "maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf" (UID: "85efe203-177e-408c-94e6-abd885b99d55") : secret "maas-api-e2e-migrate-c0006c3d-serving-cert" not found
Jul 21 04:37:04.105852 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.105833    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85efe203-177e-408c-94e6-abd885b99d55-openshift-service-ca\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.116885 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.116856    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftkgb\" (UniqueName: \"kubernetes.io/projected/85efe203-177e-408c-94e6-abd885b99d55-kube-api-access-ftkgb\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.310940 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.310899    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:04.468212 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.468177    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:04.569737 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.569653    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:04.611382 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.611350    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85efe203-177e-408c-94e6-abd885b99d55-maas-api-tls\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.613951 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.613934    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85efe203-177e-408c-94e6-abd885b99d55-maas-api-tls\") pod \"maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:04.859536 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:04.859433    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:05.006362 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:05.006332    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"]
Jul 21 04:37:05.007204 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:37:05.007181    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85efe203_177e_408c_94e6_abd885b99d55.slice/crio-2c463fdbf98853a8015f0294492734f9efd7ad78d32c525b35f3ffc0fd6cf990 WatchSource:0}: Error finding container 2c463fdbf98853a8015f0294492734f9efd7ad78d32c525b35f3ffc0fd6cf990: Status 404 returned error can't find the container with id 2c463fdbf98853a8015f0294492734f9efd7ad78d32c525b35f3ffc0fd6cf990
Jul 21 04:37:05.456542 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:05.456494    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf" event={"ID":"85efe203-177e-408c-94e6-abd885b99d55","Type":"ContainerStarted","Data":"2ac19dddaf26c2132f53656d2d72c41bbcba8ece4b86b35040c9b9a4e276751d"}
Jul 21 04:37:05.456743 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:05.456552    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf" event={"ID":"85efe203-177e-408c-94e6-abd885b99d55","Type":"ContainerStarted","Data":"2c463fdbf98853a8015f0294492734f9efd7ad78d32c525b35f3ffc0fd6cf990"}
Jul 21 04:37:05.456743 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:05.456576    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:05.475230 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:05.475185    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf" podStartSLOduration=2.132591379 podStartE2EDuration="2.475167498s" totalImagesPullingTime="342.576119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:37:05.473486421 +0000 UTC m=+1645.858975346" watchObservedRunningTime="2026-07-21 04:37:05.475167498 +0000 UTC m=+1645.860656407"
Jul 21 04:37:11.467298 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:11.467262    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:28.927259 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:28.927227    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"]
Jul 21 04:37:28.931187 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:28.931169    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"
Jul 21 04:37:28.933680 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:28.933658    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:37:28.938320 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:28.938299    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"]
Jul 21 04:37:28.958252 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:28.953856    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b515cc5c-e1cb-4100-9c84-952a3e42cadc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28\" (UID: \"b515cc5c-e1cb-4100-9c84-952a3e42cadc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"
Jul 21 04:37:29.055345 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:29.055304    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b515cc5c-e1cb-4100-9c84-952a3e42cadc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28\" (UID: \"b515cc5c-e1cb-4100-9c84-952a3e42cadc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"
Jul 21 04:37:29.055943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:29.055922    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b515cc5c-e1cb-4100-9c84-952a3e42cadc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28\" (UID: \"b515cc5c-e1cb-4100-9c84-952a3e42cadc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"
Jul 21 04:37:29.243198 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:29.243159    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"
Jul 21 04:37:29.382994 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:29.382971    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"]
Jul 21 04:37:29.385392 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:37:29.385347    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb515cc5c_e1cb_4100_9c84_952a3e42cadc.slice/crio-cbba3712179626f4a3cb342341a11dbc48d0a4b13bee1719bebff3a92f848466 WatchSource:0}: Error finding container cbba3712179626f4a3cb342341a11dbc48d0a4b13bee1719bebff3a92f848466: Status 404 returned error can't find the container with id cbba3712179626f4a3cb342341a11dbc48d0a4b13bee1719bebff3a92f848466
Jul 21 04:37:29.579223 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:29.579141    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28" event={"ID":"b515cc5c-e1cb-4100-9c84-952a3e42cadc","Type":"ContainerStarted","Data":"cbba3712179626f4a3cb342341a11dbc48d0a4b13bee1719bebff3a92f848466"}
Jul 21 04:37:30.585799 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:30.585757    2546 generic.go:356] "Generic (PLEG): container finished" podID="b515cc5c-e1cb-4100-9c84-952a3e42cadc" containerID="54e41405bcabf91a8ceacd417485568a13f931f67e174142e0d94a8cc823b046" exitCode=0
Jul 21 04:37:30.586246 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:30.585895    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28" event={"ID":"b515cc5c-e1cb-4100-9c84-952a3e42cadc","Type":"ContainerDied","Data":"54e41405bcabf91a8ceacd417485568a13f931f67e174142e0d94a8cc823b046"}
Jul 21 04:37:31.726650 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:31.726629    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"
Jul 21 04:37:31.779728 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:31.779697    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b515cc5c-e1cb-4100-9c84-952a3e42cadc-maas-api-service-ca\") pod \"b515cc5c-e1cb-4100-9c84-952a3e42cadc\" (UID: \"b515cc5c-e1cb-4100-9c84-952a3e42cadc\") "
Jul 21 04:37:31.780032 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:31.780012    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b515cc5c-e1cb-4100-9c84-952a3e42cadc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b515cc5c-e1cb-4100-9c84-952a3e42cadc" (UID: "b515cc5c-e1cb-4100-9c84-952a3e42cadc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:37:31.881274 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:31.881192    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b515cc5c-e1cb-4100-9c84-952a3e42cadc-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:37:32.597973 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:32.597943    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"
Jul 21 04:37:32.597973 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:32.597953    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28" event={"ID":"b515cc5c-e1cb-4100-9c84-952a3e42cadc","Type":"ContainerDied","Data":"cbba3712179626f4a3cb342341a11dbc48d0a4b13bee1719bebff3a92f848466"}
Jul 21 04:37:32.598212 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:32.597999    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbba3712179626f4a3cb342341a11dbc48d0a4b13bee1719bebff3a92f848466"
Jul 21 04:37:39.433834 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.433797    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"]
Jul 21 04:37:39.434214 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.434043    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf" podUID="85efe203-177e-408c-94e6-abd885b99d55" containerName="maas-api" containerID="cri-o://2ac19dddaf26c2132f53656d2d72c41bbcba8ece4b86b35040c9b9a4e276751d" gracePeriod=30
Jul 21 04:37:39.469686 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.469664    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:39.638941 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.638911    2546 generic.go:356] "Generic (PLEG): container finished" podID="85efe203-177e-408c-94e6-abd885b99d55" containerID="2ac19dddaf26c2132f53656d2d72c41bbcba8ece4b86b35040c9b9a4e276751d" exitCode=0
Jul 21 04:37:39.639106 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.639005    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf" event={"ID":"85efe203-177e-408c-94e6-abd885b99d55","Type":"ContainerDied","Data":"2ac19dddaf26c2132f53656d2d72c41bbcba8ece4b86b35040c9b9a4e276751d"}
Jul 21 04:37:39.691317 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.691254    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:39.757303 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.757274    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85efe203-177e-408c-94e6-abd885b99d55-openshift-service-ca\") pod \"85efe203-177e-408c-94e6-abd885b99d55\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") "
Jul 21 04:37:39.757447 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.757318    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85efe203-177e-408c-94e6-abd885b99d55-maas-api-tls\") pod \"85efe203-177e-408c-94e6-abd885b99d55\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") "
Jul 21 04:37:39.757447 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.757360    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftkgb\" (UniqueName: \"kubernetes.io/projected/85efe203-177e-408c-94e6-abd885b99d55-kube-api-access-ftkgb\") pod \"85efe203-177e-408c-94e6-abd885b99d55\" (UID: \"85efe203-177e-408c-94e6-abd885b99d55\") "
Jul 21 04:37:39.757635 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.757612    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85efe203-177e-408c-94e6-abd885b99d55-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "85efe203-177e-408c-94e6-abd885b99d55" (UID: "85efe203-177e-408c-94e6-abd885b99d55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:37:39.759590 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.759567    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85efe203-177e-408c-94e6-abd885b99d55-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "85efe203-177e-408c-94e6-abd885b99d55" (UID: "85efe203-177e-408c-94e6-abd885b99d55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:37:39.759692 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.759673    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85efe203-177e-408c-94e6-abd885b99d55-kube-api-access-ftkgb" (OuterVolumeSpecName: "kube-api-access-ftkgb") pod "85efe203-177e-408c-94e6-abd885b99d55" (UID: "85efe203-177e-408c-94e6-abd885b99d55"). InnerVolumeSpecName "kube-api-access-ftkgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:37:39.858126 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.858093    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85efe203-177e-408c-94e6-abd885b99d55-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:37:39.858281 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.858145    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85efe203-177e-408c-94e6-abd885b99d55-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:37:39.858281 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:39.858155    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftkgb\" (UniqueName: \"kubernetes.io/projected/85efe203-177e-408c-94e6-abd885b99d55-kube-api-access-ftkgb\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:37:40.645348 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:40.645317    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"
Jul 21 04:37:40.645348 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:40.645340    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf" event={"ID":"85efe203-177e-408c-94e6-abd885b99d55","Type":"ContainerDied","Data":"2c463fdbf98853a8015f0294492734f9efd7ad78d32c525b35f3ffc0fd6cf990"}
Jul 21 04:37:40.645862 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:40.645404    2546 scope.go:117] "RemoveContainer" containerID="2ac19dddaf26c2132f53656d2d72c41bbcba8ece4b86b35040c9b9a4e276751d"
Jul 21 04:37:40.665049 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:40.665026    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"]
Jul 21 04:37:40.669743 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:40.669720    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c0006c3d-6699b9c59b-rt7vf"]
Jul 21 04:37:42.163321 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:42.163280    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85efe203-177e-408c-94e6-abd885b99d55" path="/var/lib/kubelet/pods/85efe203-177e-408c-94e6-abd885b99d55/volumes"
Jul 21 04:37:44.014745 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:44.014716    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"]
Jul 21 04:37:44.015439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:44.014933    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:44.017743 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:44.017722    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c0006c3d-mct28"]
Jul 21 04:37:44.162816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:44.162776    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b515cc5c-e1cb-4100-9c84-952a3e42cadc" path="/var/lib/kubelet/pods/b515cc5c-e1cb-4100-9c84-952a3e42cadc/volumes"
Jul 21 04:37:44.183411 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:44.183381    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:44.438869 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:44.438764    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:44.687000 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:44.686969    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:45.027473 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:45.027439    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:45.163607 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:45.163574    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:47.734332 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.734290    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:47.834760 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.834725    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:47.895432 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.895400    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"]
Jul 21 04:37:47.895798 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.895786    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85efe203-177e-408c-94e6-abd885b99d55" containerName="maas-api"
Jul 21 04:37:47.895843 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.895799    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="85efe203-177e-408c-94e6-abd885b99d55" containerName="maas-api"
Jul 21 04:37:47.895843 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.895817    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b515cc5c-e1cb-4100-9c84-952a3e42cadc" containerName="revoke-keys"
Jul 21 04:37:47.895843 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.895823    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="b515cc5c-e1cb-4100-9c84-952a3e42cadc" containerName="revoke-keys"
Jul 21 04:37:47.895939 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.895890    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="85efe203-177e-408c-94e6-abd885b99d55" containerName="maas-api"
Jul 21 04:37:47.895939 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.895898    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="b515cc5c-e1cb-4100-9c84-952a3e42cadc" containerName="revoke-keys"
Jul 21 04:37:47.900696 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.900668    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:47.903448 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.903418    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gvc8n\""
Jul 21 04:37:47.903577 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.903452    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a261234a-serving-cert\""
Jul 21 04:37:47.908528 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.908502    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"]
Jul 21 04:37:47.936401 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:47.936348    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:48.035678 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.035602    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31e80077-4127-4b27-9264-12c391972e4d-maas-api-tls\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.035678 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.035654    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31e80077-4127-4b27-9264-12c391972e4d-openshift-service-ca\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.035856 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.035775    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfc6n\" (UniqueName: \"kubernetes.io/projected/31e80077-4127-4b27-9264-12c391972e4d-kube-api-access-gfc6n\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.136862 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.136836    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfc6n\" (UniqueName: \"kubernetes.io/projected/31e80077-4127-4b27-9264-12c391972e4d-kube-api-access-gfc6n\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.137004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.136897    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31e80077-4127-4b27-9264-12c391972e4d-maas-api-tls\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.137004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.136933    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31e80077-4127-4b27-9264-12c391972e4d-openshift-service-ca\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.137088 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:37:48.137054    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-serving-cert: secret "maas-api-e2e-ait-a261234a-serving-cert" not found
Jul 21 04:37:48.137165 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:37:48.137151    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e80077-4127-4b27-9264-12c391972e4d-maas-api-tls podName:31e80077-4127-4b27-9264-12c391972e4d nodeName:}" failed. No retries permitted until 2026-07-21 04:37:48.637099642 +0000 UTC m=+1689.022588529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/31e80077-4127-4b27-9264-12c391972e4d-maas-api-tls") pod "maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw" (UID: "31e80077-4127-4b27-9264-12c391972e4d") : secret "maas-api-e2e-ait-a261234a-serving-cert" not found
Jul 21 04:37:48.137568 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.137553    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31e80077-4127-4b27-9264-12c391972e4d-openshift-service-ca\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.150062 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.150024    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfc6n\" (UniqueName: \"kubernetes.io/projected/31e80077-4127-4b27-9264-12c391972e4d-kube-api-access-gfc6n\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.434572 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.434467    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:48.640979 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.640945    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31e80077-4127-4b27-9264-12c391972e4d-maas-api-tls\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.643527 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.643499    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31e80077-4127-4b27-9264-12c391972e4d-maas-api-tls\") pod \"maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.666746 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.666712    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:48.769690 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.769662    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:37:48.812883 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.812846    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:48.970504 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:48.970468    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"]
Jul 21 04:37:48.975263 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:37:48.975226    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e80077_4127_4b27_9264_12c391972e4d.slice/crio-6d3a817a7905ed0b7fcc9c1082b8ac746445c7c0046f8759a0a21c1dbb9a8748 WatchSource:0}: Error finding container 6d3a817a7905ed0b7fcc9c1082b8ac746445c7c0046f8759a0a21c1dbb9a8748: Status 404 returned error can't find the container with id 6d3a817a7905ed0b7fcc9c1082b8ac746445c7c0046f8759a0a21c1dbb9a8748
Jul 21 04:37:49.696178 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:49.696082    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw" event={"ID":"31e80077-4127-4b27-9264-12c391972e4d","Type":"ContainerStarted","Data":"0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797"}
Jul 21 04:37:49.696178 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:49.696155    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw" event={"ID":"31e80077-4127-4b27-9264-12c391972e4d","Type":"ContainerStarted","Data":"6d3a817a7905ed0b7fcc9c1082b8ac746445c7c0046f8759a0a21c1dbb9a8748"}
Jul 21 04:37:49.696373 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:49.696193    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:37:49.717769 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:49.717706    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw" podStartSLOduration=2.2552031120000002 podStartE2EDuration="2.717691786s" totalImagesPullingTime="462.488674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:37:49.715685222 +0000 UTC m=+1690.101174135" watchObservedRunningTime="2026-07-21 04:37:49.717691786 +0000 UTC m=+1690.103180695"
Jul 21 04:37:55.711904 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:37:55.711872    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:38:13.659067 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:13.659026    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"]
Jul 21 04:38:13.662812 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:13.662790    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"
Jul 21 04:38:13.665384 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:13.665363    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:38:13.670411 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:13.670379    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"]
Jul 21 04:38:13.775471 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:13.775441    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7123adac-a870-4dee-b653-f00024f5f6c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a261234a-9rsmf\" (UID: \"7123adac-a870-4dee-b653-f00024f5f6c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"
Jul 21 04:38:13.876536 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:13.876501    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7123adac-a870-4dee-b653-f00024f5f6c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a261234a-9rsmf\" (UID: \"7123adac-a870-4dee-b653-f00024f5f6c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"
Jul 21 04:38:13.877221 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:13.877197    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7123adac-a870-4dee-b653-f00024f5f6c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a261234a-9rsmf\" (UID: \"7123adac-a870-4dee-b653-f00024f5f6c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"
Jul 21 04:38:13.976425 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:13.976393    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"
Jul 21 04:38:14.115427 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:14.115401    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"]
Jul 21 04:38:14.117634 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:38:14.117608    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7123adac_a870_4dee_b653_f00024f5f6c4.slice/crio-94f8648b7d0801ec7936151289a723811ab96d25d9979f01f38a01768ac07f86 WatchSource:0}: Error finding container 94f8648b7d0801ec7936151289a723811ab96d25d9979f01f38a01768ac07f86: Status 404 returned error can't find the container with id 94f8648b7d0801ec7936151289a723811ab96d25d9979f01f38a01768ac07f86
Jul 21 04:38:14.840013 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:14.839984    2546 generic.go:356] "Generic (PLEG): container finished" podID="7123adac-a870-4dee-b653-f00024f5f6c4" containerID="70fd5c4485252080d539ef2fbec5701de43426f6b2a5ebc06ae7cd4539356e72" exitCode=0
Jul 21 04:38:14.840377 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:14.840033    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf" event={"ID":"7123adac-a870-4dee-b653-f00024f5f6c4","Type":"ContainerDied","Data":"70fd5c4485252080d539ef2fbec5701de43426f6b2a5ebc06ae7cd4539356e72"}
Jul 21 04:38:14.840377 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:14.840071    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf" event={"ID":"7123adac-a870-4dee-b653-f00024f5f6c4","Type":"ContainerStarted","Data":"94f8648b7d0801ec7936151289a723811ab96d25d9979f01f38a01768ac07f86"}
Jul 21 04:38:15.978244 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:15.978222    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"
Jul 21 04:38:16.093657 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:16.093623    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7123adac-a870-4dee-b653-f00024f5f6c4-maas-api-service-ca\") pod \"7123adac-a870-4dee-b653-f00024f5f6c4\" (UID: \"7123adac-a870-4dee-b653-f00024f5f6c4\") "
Jul 21 04:38:16.094039 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:16.094019    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7123adac-a870-4dee-b653-f00024f5f6c4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7123adac-a870-4dee-b653-f00024f5f6c4" (UID: "7123adac-a870-4dee-b653-f00024f5f6c4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:38:16.194646 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:16.194616    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7123adac-a870-4dee-b653-f00024f5f6c4-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:38:16.852080 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:16.852054    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"
Jul 21 04:38:16.852247 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:16.852083    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf" event={"ID":"7123adac-a870-4dee-b653-f00024f5f6c4","Type":"ContainerDied","Data":"94f8648b7d0801ec7936151289a723811ab96d25d9979f01f38a01768ac07f86"}
Jul 21 04:38:16.852247 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:16.852145    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94f8648b7d0801ec7936151289a723811ab96d25d9979f01f38a01768ac07f86"
Jul 21 04:38:24.100070 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.100037    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"]
Jul 21 04:38:24.100646 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.100339    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw" podUID="31e80077-4127-4b27-9264-12c391972e4d" containerName="maas-api" containerID="cri-o://0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797" gracePeriod=30
Jul 21 04:38:24.135065 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.135030    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:24.363691 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.363624    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:38:24.474721 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.474684    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31e80077-4127-4b27-9264-12c391972e4d-openshift-service-ca\") pod \"31e80077-4127-4b27-9264-12c391972e4d\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") "
Jul 21 04:38:24.474921 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.474753    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfc6n\" (UniqueName: \"kubernetes.io/projected/31e80077-4127-4b27-9264-12c391972e4d-kube-api-access-gfc6n\") pod \"31e80077-4127-4b27-9264-12c391972e4d\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") "
Jul 21 04:38:24.474921 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.474773    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31e80077-4127-4b27-9264-12c391972e4d-maas-api-tls\") pod \"31e80077-4127-4b27-9264-12c391972e4d\" (UID: \"31e80077-4127-4b27-9264-12c391972e4d\") "
Jul 21 04:38:24.475175 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.475150    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31e80077-4127-4b27-9264-12c391972e4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "31e80077-4127-4b27-9264-12c391972e4d" (UID: "31e80077-4127-4b27-9264-12c391972e4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:38:24.477074 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.477051    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31e80077-4127-4b27-9264-12c391972e4d-kube-api-access-gfc6n" (OuterVolumeSpecName: "kube-api-access-gfc6n") pod "31e80077-4127-4b27-9264-12c391972e4d" (UID: "31e80077-4127-4b27-9264-12c391972e4d"). InnerVolumeSpecName "kube-api-access-gfc6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:38:24.477153 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.477049    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e80077-4127-4b27-9264-12c391972e4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "31e80077-4127-4b27-9264-12c391972e4d" (UID: "31e80077-4127-4b27-9264-12c391972e4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:38:24.575791 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.575755    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31e80077-4127-4b27-9264-12c391972e4d-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:38:24.575791 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.575785    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfc6n\" (UniqueName: \"kubernetes.io/projected/31e80077-4127-4b27-9264-12c391972e4d-kube-api-access-gfc6n\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:38:24.575791 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.575795    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31e80077-4127-4b27-9264-12c391972e4d-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:38:24.893211 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.893173    2546 generic.go:356] "Generic (PLEG): container finished" podID="31e80077-4127-4b27-9264-12c391972e4d" containerID="0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797" exitCode=0
Jul 21 04:38:24.893384 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.893220    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"
Jul 21 04:38:24.893384 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.893254    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw" event={"ID":"31e80077-4127-4b27-9264-12c391972e4d","Type":"ContainerDied","Data":"0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797"}
Jul 21 04:38:24.893384 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.893307    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw" event={"ID":"31e80077-4127-4b27-9264-12c391972e4d","Type":"ContainerDied","Data":"6d3a817a7905ed0b7fcc9c1082b8ac746445c7c0046f8759a0a21c1dbb9a8748"}
Jul 21 04:38:24.893384 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.893330    2546 scope.go:117] "RemoveContainer" containerID="0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797"
Jul 21 04:38:24.904704 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.904574    2546 scope.go:117] "RemoveContainer" containerID="0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797"
Jul 21 04:38:24.905096 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:38:24.905072    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797\": container with ID starting with 0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797 not found: ID does not exist" containerID="0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797"
Jul 21 04:38:24.905239 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.905107    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797"} err="failed to get container status \"0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797\": rpc error: code = NotFound desc = could not find container \"0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797\": container with ID starting with 0cf6f2d81d4415e2d78c35164dafb18df367af79c6af1f70b74f288ea91b2797 not found: ID does not exist"
Jul 21 04:38:24.916701 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.916678    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"]
Jul 21 04:38:24.920562 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:24.920542    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a261234a-c4c8d8c6b-tzxcw"]
Jul 21 04:38:26.164130 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:26.164087    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31e80077-4127-4b27-9264-12c391972e4d" path="/var/lib/kubelet/pods/31e80077-4127-4b27-9264-12c391972e4d/volumes"
Jul 21 04:38:26.583016 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:26.582969    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:28.737899 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:28.737830    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"]
Jul 21 04:38:28.738380 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:28.738053    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:28.743415 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:28.743392    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a261234a-9rsmf"]
Jul 21 04:38:29.976685 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:29.976648    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:30.163031 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:30.162997    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7123adac-a870-4dee-b653-f00024f5f6c4" path="/var/lib/kubelet/pods/7123adac-a870-4dee-b653-f00024f5f6c4/volumes"
Jul 21 04:38:30.225766 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:30.225734    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:30.448348 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:30.448263    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:30.768483 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:30.768445    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:31.100974 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:31.100880    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:33.260956 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.260914    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:33.359188 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.359154    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:33.493812 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.493776    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"]
Jul 21 04:38:33.494357 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.494334    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7123adac-a870-4dee-b653-f00024f5f6c4" containerName="revoke-keys"
Jul 21 04:38:33.494357 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.494354    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="7123adac-a870-4dee-b653-f00024f5f6c4" containerName="revoke-keys"
Jul 21 04:38:33.494526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.494372    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31e80077-4127-4b27-9264-12c391972e4d" containerName="maas-api"
Jul 21 04:38:33.494526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.494378    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="31e80077-4127-4b27-9264-12c391972e4d" containerName="maas-api"
Jul 21 04:38:33.494526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.494455    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="7123adac-a870-4dee-b653-f00024f5f6c4" containerName="revoke-keys"
Jul 21 04:38:33.494526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.494467    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="31e80077-4127-4b27-9264-12c391972e4d" containerName="maas-api"
Jul 21 04:38:33.498901 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.498882    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.501654 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.501635    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-dfb01d92-serving-cert\""
Jul 21 04:38:33.501840 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.501823    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gvc8n\""
Jul 21 04:38:33.507872 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.507846    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"]
Jul 21 04:38:33.529709 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.529615    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:33.666166 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.666138    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69d72e7-db30-41e6-9916-17b689501bc9-openshift-service-ca\") pod \"maas-api-e2e-derive-dfb01d92-699b59d859-wkm29\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.666166 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.666173    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84p4c\" (UniqueName: \"kubernetes.io/projected/d69d72e7-db30-41e6-9916-17b689501bc9-kube-api-access-84p4c\") pod \"maas-api-e2e-derive-dfb01d92-699b59d859-wkm29\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.666370 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.666222    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d69d72e7-db30-41e6-9916-17b689501bc9-maas-api-tls\") pod \"maas-api-e2e-derive-dfb01d92-699b59d859-wkm29\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.766970 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.766935    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69d72e7-db30-41e6-9916-17b689501bc9-openshift-service-ca\") pod \"maas-api-e2e-derive-dfb01d92-699b59d859-wkm29\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.766970 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.766970    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84p4c\" (UniqueName: \"kubernetes.io/projected/d69d72e7-db30-41e6-9916-17b689501bc9-kube-api-access-84p4c\") pod \"maas-api-e2e-derive-dfb01d92-699b59d859-wkm29\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.767268 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.766998    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d69d72e7-db30-41e6-9916-17b689501bc9-maas-api-tls\") pod \"maas-api-e2e-derive-dfb01d92-699b59d859-wkm29\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.767649 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.767621    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69d72e7-db30-41e6-9916-17b689501bc9-openshift-service-ca\") pod \"maas-api-e2e-derive-dfb01d92-699b59d859-wkm29\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.769556 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.769534    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d69d72e7-db30-41e6-9916-17b689501bc9-maas-api-tls\") pod \"maas-api-e2e-derive-dfb01d92-699b59d859-wkm29\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.776594 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.776563    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84p4c\" (UniqueName: \"kubernetes.io/projected/d69d72e7-db30-41e6-9916-17b689501bc9-kube-api-access-84p4c\") pod \"maas-api-e2e-derive-dfb01d92-699b59d859-wkm29\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.808325 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.808236    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:33.811212 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.811196    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:33.946316 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:33.946294    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"]
Jul 21 04:38:33.948893 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:38:33.948863    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69d72e7_db30_41e6_9916_17b689501bc9.slice/crio-1a25548bdcc267dce201fc05bebae4f57699cd786398d68e6e30792a33cafde1 WatchSource:0}: Error finding container 1a25548bdcc267dce201fc05bebae4f57699cd786398d68e6e30792a33cafde1: Status 404 returned error can't find the container with id 1a25548bdcc267dce201fc05bebae4f57699cd786398d68e6e30792a33cafde1
Jul 21 04:38:34.079910 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:34.079825    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:34.179453 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:34.179406    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:34.948159 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:34.948107    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29" event={"ID":"d69d72e7-db30-41e6-9916-17b689501bc9","Type":"ContainerStarted","Data":"f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb"}
Jul 21 04:38:34.948602 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:34.948180    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29" event={"ID":"d69d72e7-db30-41e6-9916-17b689501bc9","Type":"ContainerStarted","Data":"1a25548bdcc267dce201fc05bebae4f57699cd786398d68e6e30792a33cafde1"}
Jul 21 04:38:34.948602 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:34.948270    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:34.967481 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:34.967438    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29" podStartSLOduration=1.544055793 podStartE2EDuration="1.967422834s" totalImagesPullingTime="423.367041ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:38:34.966749432 +0000 UTC m=+1735.352238344" watchObservedRunningTime="2026-07-21 04:38:34.967422834 +0000 UTC m=+1735.352911742"
Jul 21 04:38:40.960660 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:40.960626    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:38:46.250187 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:46.250143    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:46.350368 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:46.350332    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:56.582461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:56.582422    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:56.685182 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:56.685148    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:38:58.216465 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:58.216430    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"]
Jul 21 04:38:58.221479 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:58.221460    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"
Jul 21 04:38:58.224422 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:58.224390    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:38:58.228605 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:58.228582    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"]
Jul 21 04:38:58.301049 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:58.301017    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb99260f-3e9a-4403-980b-7146b7bc1dd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr\" (UID: \"fb99260f-3e9a-4403-980b-7146b7bc1dd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"
Jul 21 04:38:58.402521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:58.402490    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb99260f-3e9a-4403-980b-7146b7bc1dd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr\" (UID: \"fb99260f-3e9a-4403-980b-7146b7bc1dd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"
Jul 21 04:38:58.403095 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:58.403077    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb99260f-3e9a-4403-980b-7146b7bc1dd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr\" (UID: \"fb99260f-3e9a-4403-980b-7146b7bc1dd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"
Jul 21 04:38:58.535248 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:58.535214    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"
Jul 21 04:38:58.678007 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:58.677979    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"]
Jul 21 04:38:58.680524 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:38:58.680495    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb99260f_3e9a_4403_980b_7146b7bc1dd5.slice/crio-1778782d820a27e3934f36c035c084ae5ad3c0d1bc711e4366e75389b98defdb WatchSource:0}: Error finding container 1778782d820a27e3934f36c035c084ae5ad3c0d1bc711e4366e75389b98defdb: Status 404 returned error can't find the container with id 1778782d820a27e3934f36c035c084ae5ad3c0d1bc711e4366e75389b98defdb
Jul 21 04:38:59.082436 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:59.082403    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr" event={"ID":"fb99260f-3e9a-4403-980b-7146b7bc1dd5","Type":"ContainerStarted","Data":"38cb3975e302132e840d2d1682b10d67c6061dab2fd8659858237f0a9f26c086"}
Jul 21 04:38:59.082603 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:59.082455    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr" event={"ID":"fb99260f-3e9a-4403-980b-7146b7bc1dd5","Type":"ContainerStarted","Data":"1778782d820a27e3934f36c035c084ae5ad3c0d1bc711e4366e75389b98defdb"}
Jul 21 04:38:59.100442 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:38:59.100387    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr" podStartSLOduration=1.100370371 podStartE2EDuration="1.100370371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:38:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:38:59.099784201 +0000 UTC m=+1759.485273111" watchObservedRunningTime="2026-07-21 04:38:59.100370371 +0000 UTC m=+1759.485859278"
Jul 21 04:39:00.089087 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:00.089041    2546 generic.go:356] "Generic (PLEG): container finished" podID="fb99260f-3e9a-4403-980b-7146b7bc1dd5" containerID="38cb3975e302132e840d2d1682b10d67c6061dab2fd8659858237f0a9f26c086" exitCode=0
Jul 21 04:39:00.089485 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:00.089156    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr" event={"ID":"fb99260f-3e9a-4403-980b-7146b7bc1dd5","Type":"ContainerDied","Data":"38cb3975e302132e840d2d1682b10d67c6061dab2fd8659858237f0a9f26c086"}
Jul 21 04:39:01.233732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:01.233708    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"
Jul 21 04:39:01.330386 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:01.330349    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb99260f-3e9a-4403-980b-7146b7bc1dd5-maas-api-service-ca\") pod \"fb99260f-3e9a-4403-980b-7146b7bc1dd5\" (UID: \"fb99260f-3e9a-4403-980b-7146b7bc1dd5\") "
Jul 21 04:39:01.330718 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:01.330693    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb99260f-3e9a-4403-980b-7146b7bc1dd5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fb99260f-3e9a-4403-980b-7146b7bc1dd5" (UID: "fb99260f-3e9a-4403-980b-7146b7bc1dd5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:39:01.431766 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:01.431680    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb99260f-3e9a-4403-980b-7146b7bc1dd5-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:39:02.101892 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:02.101859    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr" event={"ID":"fb99260f-3e9a-4403-980b-7146b7bc1dd5","Type":"ContainerDied","Data":"1778782d820a27e3934f36c035c084ae5ad3c0d1bc711e4366e75389b98defdb"}
Jul 21 04:39:02.102064 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:02.101939    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1778782d820a27e3934f36c035c084ae5ad3c0d1bc711e4366e75389b98defdb"
Jul 21 04:39:02.102064 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:02.101871    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"
Jul 21 04:39:08.708099 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:08.708060    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"]
Jul 21 04:39:08.708676 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:08.708354    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29" podUID="d69d72e7-db30-41e6-9916-17b689501bc9" containerName="maas-api" containerID="cri-o://f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb" gracePeriod=30
Jul 21 04:39:08.729461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:08.729429    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:08.958967 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:08.958909    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:39:09.124625 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.124587    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69d72e7-db30-41e6-9916-17b689501bc9-openshift-service-ca\") pod \"d69d72e7-db30-41e6-9916-17b689501bc9\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") "
Jul 21 04:39:09.124798 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.124657    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84p4c\" (UniqueName: \"kubernetes.io/projected/d69d72e7-db30-41e6-9916-17b689501bc9-kube-api-access-84p4c\") pod \"d69d72e7-db30-41e6-9916-17b689501bc9\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") "
Jul 21 04:39:09.124798 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.124757    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d69d72e7-db30-41e6-9916-17b689501bc9-maas-api-tls\") pod \"d69d72e7-db30-41e6-9916-17b689501bc9\" (UID: \"d69d72e7-db30-41e6-9916-17b689501bc9\") "
Jul 21 04:39:09.124985 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.124957    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d69d72e7-db30-41e6-9916-17b689501bc9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d69d72e7-db30-41e6-9916-17b689501bc9" (UID: "d69d72e7-db30-41e6-9916-17b689501bc9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:39:09.126876 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.126843    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d69d72e7-db30-41e6-9916-17b689501bc9-kube-api-access-84p4c" (OuterVolumeSpecName: "kube-api-access-84p4c") pod "d69d72e7-db30-41e6-9916-17b689501bc9" (UID: "d69d72e7-db30-41e6-9916-17b689501bc9"). InnerVolumeSpecName "kube-api-access-84p4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:39:09.126980 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.126952    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d69d72e7-db30-41e6-9916-17b689501bc9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d69d72e7-db30-41e6-9916-17b689501bc9" (UID: "d69d72e7-db30-41e6-9916-17b689501bc9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:39:09.141044 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.141008    2546 generic.go:356] "Generic (PLEG): container finished" podID="d69d72e7-db30-41e6-9916-17b689501bc9" containerID="f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb" exitCode=0
Jul 21 04:39:09.141219 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.141054    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"
Jul 21 04:39:09.141219 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.141087    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29" event={"ID":"d69d72e7-db30-41e6-9916-17b689501bc9","Type":"ContainerDied","Data":"f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb"}
Jul 21 04:39:09.141219 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.141177    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29" event={"ID":"d69d72e7-db30-41e6-9916-17b689501bc9","Type":"ContainerDied","Data":"1a25548bdcc267dce201fc05bebae4f57699cd786398d68e6e30792a33cafde1"}
Jul 21 04:39:09.141219 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.141213    2546 scope.go:117] "RemoveContainer" containerID="f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb"
Jul 21 04:39:09.151860 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.151839    2546 scope.go:117] "RemoveContainer" containerID="f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb"
Jul 21 04:39:09.152431 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:39:09.152404    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb\": container with ID starting with f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb not found: ID does not exist" containerID="f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb"
Jul 21 04:39:09.152521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.152439    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb"} err="failed to get container status \"f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb\": rpc error: code = NotFound desc = could not find container \"f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb\": container with ID starting with f5469e960c3072186eb4065a86550897ef0329bb4eceeee62c89aa7819f385fb not found: ID does not exist"
Jul 21 04:39:09.165264 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.165241    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"]
Jul 21 04:39:09.167454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.167433    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dfb01d92-699b59d859-wkm29"]
Jul 21 04:39:09.225700 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.225675    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84p4c\" (UniqueName: \"kubernetes.io/projected/d69d72e7-db30-41e6-9916-17b689501bc9-kube-api-access-84p4c\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:39:09.225846 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.225701    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d69d72e7-db30-41e6-9916-17b689501bc9-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:39:09.225846 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:09.225720    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69d72e7-db30-41e6-9916-17b689501bc9-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:39:10.162314 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:10.162282    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d69d72e7-db30-41e6-9916-17b689501bc9" path="/var/lib/kubelet/pods/d69d72e7-db30-41e6-9916-17b689501bc9/volumes"
Jul 21 04:39:13.290351 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:13.290314    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"]
Jul 21 04:39:13.292934 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:13.292909    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-dfb01d92-4tbpr"]
Jul 21 04:39:13.294142 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:13.294107    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:13.454620 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:13.454584    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:13.708423 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:13.708332    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:14.164217 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:14.164186    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb99260f-3e9a-4403-980b-7146b7bc1dd5" path="/var/lib/kubelet/pods/fb99260f-3e9a-4403-980b-7146b7bc1dd5/volumes"
Jul 21 04:39:14.196069 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:14.196035    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:14.295830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:14.295795    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:14.400467 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:14.400439    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:15.855265 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:15.855223    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:22.566476 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:22.566446    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:22.675892 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:22.675856    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:22.770666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:22.770636    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:22.866776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:22.866691    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:28.433078 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:28.433027    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:39:40.184669 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:40.184644    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:39:40.189781 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:40.189759    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:39:54.415460 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:39:54.415429    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:01.015599 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:01.015549    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:01.120061 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:01.120026    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:01.215871 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:01.215829    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:17.161436 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:17.161401    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:23.120704 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:23.120671    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:23.181857 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:23.181824    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:23.265707 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:23.265675    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:23.368110 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:23.368073    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:33.878177 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:33.878104    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:34.698456 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:34.698402    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:41.215653 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:41.215621    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:41.318018 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:41.317988    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:41.416708 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:41.416674    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:41.521723 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:41.521698    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:42.302981 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:42.302949    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:42.407295 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:42.407264    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:42.508280 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:42.508253    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:44.131951 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:44.131914    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:45.280002 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:45.279966    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:45.338707 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:45.338675    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:50.225728 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:50.225692    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:50.325558 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:50.325515    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:50.906173 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:50.906141    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:57.184958 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:57.184926    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:57.284666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:57.284631    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:57.432725 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:57.432682    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:57.766538 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:57.766498    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:40:58.125616 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:40:58.125514    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:00.708199 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.708084    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:00.813957 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.813926    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:00.847909 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.847880    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"]
Jul 21 04:41:00.848362 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.848342    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69d72e7-db30-41e6-9916-17b689501bc9" containerName="maas-api"
Jul 21 04:41:00.848362 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.848364    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69d72e7-db30-41e6-9916-17b689501bc9" containerName="maas-api"
Jul 21 04:41:00.848535 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.848397    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb99260f-3e9a-4403-980b-7146b7bc1dd5" containerName="revoke-keys"
Jul 21 04:41:00.848535 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.848404    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb99260f-3e9a-4403-980b-7146b7bc1dd5" containerName="revoke-keys"
Jul 21 04:41:00.848535 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.848464    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="d69d72e7-db30-41e6-9916-17b689501bc9" containerName="maas-api"
Jul 21 04:41:00.848535 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.848476    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb99260f-3e9a-4403-980b-7146b7bc1dd5" containerName="revoke-keys"
Jul 21 04:41:00.851883 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.851863    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:00.854254 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.854227    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6f1bb607-serving-cert\""
Jul 21 04:41:00.854369 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.854277    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gvc8n\""
Jul 21 04:41:00.861979 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.861901    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"]
Jul 21 04:41:00.914862 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.914834    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:00.960397 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.960307    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfqx7\" (UniqueName: \"kubernetes.io/projected/ea0085d0-6ae7-4eff-aa79-198a2cff446c-kube-api-access-sfqx7\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:00.960397 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.960363    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0085d0-6ae7-4eff-aa79-198a2cff446c-maas-api-tls\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:00.960584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:00.960472    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0085d0-6ae7-4eff-aa79-198a2cff446c-openshift-service-ca\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:01.061270 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.061233    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfqx7\" (UniqueName: \"kubernetes.io/projected/ea0085d0-6ae7-4eff-aa79-198a2cff446c-kube-api-access-sfqx7\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:01.061454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.061279    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0085d0-6ae7-4eff-aa79-198a2cff446c-maas-api-tls\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:01.061454 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:41:01.061382    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-serving-cert: secret "maas-api-e2e-mt-6f1bb607-serving-cert" not found
Jul 21 04:41:01.061454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.061408    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0085d0-6ae7-4eff-aa79-198a2cff446c-openshift-service-ca\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:01.061454 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:41:01.061437    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea0085d0-6ae7-4eff-aa79-198a2cff446c-maas-api-tls podName:ea0085d0-6ae7-4eff-aa79-198a2cff446c nodeName:}" failed. No retries permitted until 2026-07-21 04:41:01.561420785 +0000 UTC m=+1881.946909673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea0085d0-6ae7-4eff-aa79-198a2cff446c-maas-api-tls") pod "maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72" (UID: "ea0085d0-6ae7-4eff-aa79-198a2cff446c") : secret "maas-api-e2e-mt-6f1bb607-serving-cert" not found
Jul 21 04:41:01.061958 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.061940    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0085d0-6ae7-4eff-aa79-198a2cff446c-openshift-service-ca\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:01.074939 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.074921    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfqx7\" (UniqueName: \"kubernetes.io/projected/ea0085d0-6ae7-4eff-aa79-198a2cff446c-kube-api-access-sfqx7\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:01.208377 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.208340    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:01.380454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.380426    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:01.480723 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.480693    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:01.566197 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.566165    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0085d0-6ae7-4eff-aa79-198a2cff446c-maas-api-tls\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:01.568799 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.568778    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0085d0-6ae7-4eff-aa79-198a2cff446c-maas-api-tls\") pod \"maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:01.763885 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.763845    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:01.905027 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:01.905004    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"]
Jul 21 04:41:01.907386 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:41:01.907343    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea0085d0_6ae7_4eff_aa79_198a2cff446c.slice/crio-fee56d030a831ea544e211ef2b00b0152a9d689da2ccc54f658381bdc426087a WatchSource:0}: Error finding container fee56d030a831ea544e211ef2b00b0152a9d689da2ccc54f658381bdc426087a: Status 404 returned error can't find the container with id fee56d030a831ea544e211ef2b00b0152a9d689da2ccc54f658381bdc426087a
Jul 21 04:41:02.734546 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:02.734512    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72" event={"ID":"ea0085d0-6ae7-4eff-aa79-198a2cff446c","Type":"ContainerStarted","Data":"0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397"}
Jul 21 04:41:02.734748 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:02.734563    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72" event={"ID":"ea0085d0-6ae7-4eff-aa79-198a2cff446c","Type":"ContainerStarted","Data":"fee56d030a831ea544e211ef2b00b0152a9d689da2ccc54f658381bdc426087a"}
Jul 21 04:41:02.734748 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:02.734584    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:02.752947 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:02.752900    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72" podStartSLOduration=2.361695471 podStartE2EDuration="2.752885431s" totalImagesPullingTime="391.18996ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:41:02.752533748 +0000 UTC m=+1883.138022658" watchObservedRunningTime="2026-07-21 04:41:02.752885431 +0000 UTC m=+1883.138374340"
Jul 21 04:41:05.240959 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:05.240913    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:05.918575 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:05.918531    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:06.010836 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:06.010805    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:06.353942 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:06.353880    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:08.745323 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:08.745297    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:12.397060 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:12.397011    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:12.462385 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:12.462355    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:12.756820 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:12.756786    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"]
Jul 21 04:41:12.760443 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:12.760428    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"
Jul 21 04:41:12.762860 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:12.762839    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:41:12.767675 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:12.767654    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"]
Jul 21 04:41:12.873281 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:12.873249    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5229cc-2ae7-4165-8871-c5baba2df0b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk\" (UID: \"fe5229cc-2ae7-4165-8871-c5baba2df0b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"
Jul 21 04:41:12.974065 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:12.974030    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5229cc-2ae7-4165-8871-c5baba2df0b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk\" (UID: \"fe5229cc-2ae7-4165-8871-c5baba2df0b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"
Jul 21 04:41:12.974639 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:12.974620    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5229cc-2ae7-4165-8871-c5baba2df0b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk\" (UID: \"fe5229cc-2ae7-4165-8871-c5baba2df0b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"
Jul 21 04:41:13.074166 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:13.074053    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"
Jul 21 04:41:13.215511 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:13.215473    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"]
Jul 21 04:41:13.219655 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:41:13.219624    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe5229cc_2ae7_4165_8871_c5baba2df0b1.slice/crio-9e379c8412ea5701189528ccd948688909cab42144bcc9b2da306ef156907034 WatchSource:0}: Error finding container 9e379c8412ea5701189528ccd948688909cab42144bcc9b2da306ef156907034: Status 404 returned error can't find the container with id 9e379c8412ea5701189528ccd948688909cab42144bcc9b2da306ef156907034
Jul 21 04:41:13.795409 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:13.795372    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk" event={"ID":"fe5229cc-2ae7-4165-8871-c5baba2df0b1","Type":"ContainerStarted","Data":"f31000bb6e4cfa2c1ece9ed6ffcd1e525ecb1d8507f2689fb4a3e4bb99c763ad"}
Jul 21 04:41:13.795795 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:13.795437    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk" event={"ID":"fe5229cc-2ae7-4165-8871-c5baba2df0b1","Type":"ContainerStarted","Data":"9e379c8412ea5701189528ccd948688909cab42144bcc9b2da306ef156907034"}
Jul 21 04:41:13.813767 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:13.813718    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk" podStartSLOduration=1.813703983 podStartE2EDuration="1.813703983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:41:13.812041275 +0000 UTC m=+1894.197530387" watchObservedRunningTime="2026-07-21 04:41:13.813703983 +0000 UTC m=+1894.199192892"
Jul 21 04:41:14.802169 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:14.802136    2546 generic.go:356] "Generic (PLEG): container finished" podID="fe5229cc-2ae7-4165-8871-c5baba2df0b1" containerID="f31000bb6e4cfa2c1ece9ed6ffcd1e525ecb1d8507f2689fb4a3e4bb99c763ad" exitCode=0
Jul 21 04:41:14.802534 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:14.802205    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk" event={"ID":"fe5229cc-2ae7-4165-8871-c5baba2df0b1","Type":"ContainerDied","Data":"f31000bb6e4cfa2c1ece9ed6ffcd1e525ecb1d8507f2689fb4a3e4bb99c763ad"}
Jul 21 04:41:15.950921 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:15.950896    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"
Jul 21 04:41:16.101298 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:16.101205    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5229cc-2ae7-4165-8871-c5baba2df0b1-maas-api-service-ca\") pod \"fe5229cc-2ae7-4165-8871-c5baba2df0b1\" (UID: \"fe5229cc-2ae7-4165-8871-c5baba2df0b1\") "
Jul 21 04:41:16.101596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:16.101569    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe5229cc-2ae7-4165-8871-c5baba2df0b1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fe5229cc-2ae7-4165-8871-c5baba2df0b1" (UID: "fe5229cc-2ae7-4165-8871-c5baba2df0b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:41:16.202692 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:16.202663    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5229cc-2ae7-4165-8871-c5baba2df0b1-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:41:16.814036 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:16.814005    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"
Jul 21 04:41:16.814223 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:16.814037    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk" event={"ID":"fe5229cc-2ae7-4165-8871-c5baba2df0b1","Type":"ContainerDied","Data":"9e379c8412ea5701189528ccd948688909cab42144bcc9b2da306ef156907034"}
Jul 21 04:41:16.814223 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:16.814084    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e379c8412ea5701189528ccd948688909cab42144bcc9b2da306ef156907034"
Jul 21 04:41:23.144234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.144204    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"]
Jul 21 04:41:23.144630 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.144414    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72" podUID="ea0085d0-6ae7-4eff-aa79-198a2cff446c" containerName="maas-api" containerID="cri-o://0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397" gracePeriod=30
Jul 21 04:41:23.171029 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.171004    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:23.398862 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.398804    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:23.570568 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.570530    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfqx7\" (UniqueName: \"kubernetes.io/projected/ea0085d0-6ae7-4eff-aa79-198a2cff446c-kube-api-access-sfqx7\") pod \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") "
Jul 21 04:41:23.570568 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.570569    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0085d0-6ae7-4eff-aa79-198a2cff446c-maas-api-tls\") pod \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") "
Jul 21 04:41:23.570821 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.570675    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0085d0-6ae7-4eff-aa79-198a2cff446c-openshift-service-ca\") pod \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\" (UID: \"ea0085d0-6ae7-4eff-aa79-198a2cff446c\") "
Jul 21 04:41:23.571047 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.571023    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea0085d0-6ae7-4eff-aa79-198a2cff446c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea0085d0-6ae7-4eff-aa79-198a2cff446c" (UID: "ea0085d0-6ae7-4eff-aa79-198a2cff446c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:41:23.572753 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.572722    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea0085d0-6ae7-4eff-aa79-198a2cff446c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea0085d0-6ae7-4eff-aa79-198a2cff446c" (UID: "ea0085d0-6ae7-4eff-aa79-198a2cff446c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:41:23.572861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.572767    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea0085d0-6ae7-4eff-aa79-198a2cff446c-kube-api-access-sfqx7" (OuterVolumeSpecName: "kube-api-access-sfqx7") pod "ea0085d0-6ae7-4eff-aa79-198a2cff446c" (UID: "ea0085d0-6ae7-4eff-aa79-198a2cff446c"). InnerVolumeSpecName "kube-api-access-sfqx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:41:23.671955 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.671887    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfqx7\" (UniqueName: \"kubernetes.io/projected/ea0085d0-6ae7-4eff-aa79-198a2cff446c-kube-api-access-sfqx7\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:41:23.671955 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.671911    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0085d0-6ae7-4eff-aa79-198a2cff446c-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:41:23.671955 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.671922    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0085d0-6ae7-4eff-aa79-198a2cff446c-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:41:23.852682 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.852650    2546 generic.go:356] "Generic (PLEG): container finished" podID="ea0085d0-6ae7-4eff-aa79-198a2cff446c" containerID="0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397" exitCode=0
Jul 21 04:41:23.852830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.852688    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"
Jul 21 04:41:23.852830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.852725    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72" event={"ID":"ea0085d0-6ae7-4eff-aa79-198a2cff446c","Type":"ContainerDied","Data":"0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397"}
Jul 21 04:41:23.852830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.852763    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72" event={"ID":"ea0085d0-6ae7-4eff-aa79-198a2cff446c","Type":"ContainerDied","Data":"fee56d030a831ea544e211ef2b00b0152a9d689da2ccc54f658381bdc426087a"}
Jul 21 04:41:23.852830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.852784    2546 scope.go:117] "RemoveContainer" containerID="0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397"
Jul 21 04:41:23.864722 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.864685    2546 scope.go:117] "RemoveContainer" containerID="0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397"
Jul 21 04:41:23.865052 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:41:23.865029    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397\": container with ID starting with 0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397 not found: ID does not exist" containerID="0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397"
Jul 21 04:41:23.865295 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.865057    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397"} err="failed to get container status \"0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397\": rpc error: code = NotFound desc = could not find container \"0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397\": container with ID starting with 0f26d6858a6054b6bde2fa1e06f6fc23bb114890dac8b706d1f560e90733d397 not found: ID does not exist"
Jul 21 04:41:23.875914 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.875889    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"]
Jul 21 04:41:23.880269 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:23.880249    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f1bb607-75f46f7bcc-8sn72"]
Jul 21 04:41:24.163027 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:24.162990    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea0085d0-6ae7-4eff-aa79-198a2cff446c" path="/var/lib/kubelet/pods/ea0085d0-6ae7-4eff-aa79-198a2cff446c/volumes"
Jul 21 04:41:27.829655 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:27.829615    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:27.839940 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:27.839916    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"]
Jul 21 04:41:27.845396 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:27.845376    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f1bb607-pr2tk"]
Jul 21 04:41:28.163540 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:28.163469    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe5229cc-2ae7-4165-8871-c5baba2df0b1" path="/var/lib/kubelet/pods/fe5229cc-2ae7-4165-8871-c5baba2df0b1/volumes"
Jul 21 04:41:28.208969 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:28.208918    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:34.011732 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:34.011680    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:34.071534 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:34.071507    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:34.275484 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:34.275396    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:34.501170 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:34.501104    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:34.616349 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:34.616275    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:34.718741 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:34.718711    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:34.838725 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:34.838698    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:40.907764 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:40.907729    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:41.007865 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:41.007822    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:41.117830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:41.117801    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:41.238629 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:41.238591    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:47.180952 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:47.180914    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:47.281367 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:47.281331    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:54.013974 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:54.013939    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:54.367795 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:54.367679    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:54.505374 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:54.505341    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:57.068874 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.068839    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:57.166585 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.166559    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:57.179507 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.179483    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"]
Jul 21 04:41:57.179905 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.179891    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe5229cc-2ae7-4165-8871-c5baba2df0b1" containerName="revoke-keys"
Jul 21 04:41:57.179905 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.179905    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe5229cc-2ae7-4165-8871-c5baba2df0b1" containerName="revoke-keys"
Jul 21 04:41:57.180073 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.179915    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea0085d0-6ae7-4eff-aa79-198a2cff446c" containerName="maas-api"
Jul 21 04:41:57.180073 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.179922    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea0085d0-6ae7-4eff-aa79-198a2cff446c" containerName="maas-api"
Jul 21 04:41:57.180073 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.179978    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe5229cc-2ae7-4165-8871-c5baba2df0b1" containerName="revoke-keys"
Jul 21 04:41:57.180073 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.179985    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea0085d0-6ae7-4eff-aa79-198a2cff446c" containerName="maas-api"
Jul 21 04:41:57.183292 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.183273    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.185881 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.185863    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6ae222fa-serving-cert\""
Jul 21 04:41:57.185973 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.185865    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gvc8n\""
Jul 21 04:41:57.192807 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.192780    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"]
Jul 21 04:41:57.260550 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.260518    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade1de17-bbbe-4c4a-88d9-debec940af4c-maas-api-tls\") pod \"maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.260688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.260562    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade1de17-bbbe-4c4a-88d9-debec940af4c-openshift-service-ca\") pod \"maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.260688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.260618    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtm57\" (UniqueName: \"kubernetes.io/projected/ade1de17-bbbe-4c4a-88d9-debec940af4c-kube-api-access-mtm57\") pod \"maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.263484 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.263456    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:57.361299 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.361203    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade1de17-bbbe-4c4a-88d9-debec940af4c-maas-api-tls\") pod \"maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.361299 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.361240    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade1de17-bbbe-4c4a-88d9-debec940af4c-openshift-service-ca\") pod \"maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.361299 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.361268    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtm57\" (UniqueName: \"kubernetes.io/projected/ade1de17-bbbe-4c4a-88d9-debec940af4c-kube-api-access-mtm57\") pod \"maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.361916 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.361884    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:57.362234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.362036    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade1de17-bbbe-4c4a-88d9-debec940af4c-openshift-service-ca\") pod \"maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.363905 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.363883    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade1de17-bbbe-4c4a-88d9-debec940af4c-maas-api-tls\") pod \"maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.370131 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.370098    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtm57\" (UniqueName: \"kubernetes.io/projected/ade1de17-bbbe-4c4a-88d9-debec940af4c-kube-api-access-mtm57\") pod \"maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.495356 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.495334    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:57.641541 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.641512    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"]
Jul 21 04:41:57.643945 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:41:57.643918    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade1de17_bbbe_4c4a_88d9_debec940af4c.slice/crio-5f0f938338941808436f23d14fa1721d23d41a3b3afacd248977b5593fd91bc2 WatchSource:0}: Error finding container 5f0f938338941808436f23d14fa1721d23d41a3b3afacd248977b5593fd91bc2: Status 404 returned error can't find the container with id 5f0f938338941808436f23d14fa1721d23d41a3b3afacd248977b5593fd91bc2
Jul 21 04:41:57.645561 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.645537    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 04:41:57.727768 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.727731    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:57.828246 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:57.828216    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:41:58.033329 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:58.033289    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl" event={"ID":"ade1de17-bbbe-4c4a-88d9-debec940af4c","Type":"ContainerStarted","Data":"5f0f938338941808436f23d14fa1721d23d41a3b3afacd248977b5593fd91bc2"}
Jul 21 04:41:59.041275 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:59.041233    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl" event={"ID":"ade1de17-bbbe-4c4a-88d9-debec940af4c","Type":"ContainerStarted","Data":"0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242"}
Jul 21 04:41:59.041670 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:59.041306    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:41:59.059297 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:41:59.059254    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl" podStartSLOduration=1.562234758 podStartE2EDuration="2.059239496s" totalImagesPullingTime="497.004738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:41:59.058043282 +0000 UTC m=+1939.443532186" watchObservedRunningTime="2026-07-21 04:41:59.059239496 +0000 UTC m=+1939.444728404"
Jul 21 04:42:01.609134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:01.609081    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:42:05.051958 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:05.051929    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:42:17.192352 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:17.192303    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:42:45.350320 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:45.350227    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"]
Jul 21 04:42:45.353785 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:45.353766    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"
Jul 21 04:42:45.356234 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:45.356217    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:42:45.361265 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:45.361242    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"]
Jul 21 04:42:45.496928 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:45.496899    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0b4b5f0-2f6e-4916-8278-ecf886457a6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j\" (UID: \"b0b4b5f0-2f6e-4916-8278-ecf886457a6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"
Jul 21 04:42:45.597456 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:45.597417    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0b4b5f0-2f6e-4916-8278-ecf886457a6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j\" (UID: \"b0b4b5f0-2f6e-4916-8278-ecf886457a6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"
Jul 21 04:42:45.598060 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:45.598039    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0b4b5f0-2f6e-4916-8278-ecf886457a6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j\" (UID: \"b0b4b5f0-2f6e-4916-8278-ecf886457a6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"
Jul 21 04:42:45.665261 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:45.665170    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"
Jul 21 04:42:45.807612 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:45.807584    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"]
Jul 21 04:42:45.811715 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:42:45.811692    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b4b5f0_2f6e_4916_8278_ecf886457a6b.slice/crio-82e331e9e0db6b7dc7bc171ff1013a0269829124c290d598781ab4c331d2f74f WatchSource:0}: Error finding container 82e331e9e0db6b7dc7bc171ff1013a0269829124c290d598781ab4c331d2f74f: Status 404 returned error can't find the container with id 82e331e9e0db6b7dc7bc171ff1013a0269829124c290d598781ab4c331d2f74f
Jul 21 04:42:46.294501 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:46.294396    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j" event={"ID":"b0b4b5f0-2f6e-4916-8278-ecf886457a6b","Type":"ContainerStarted","Data":"b055fc201c89069b3f3f5d487f1c61026bcc21c1f2327dd9a27b7deb8e5bbeef"}
Jul 21 04:42:46.294501 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:46.294488    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j" event={"ID":"b0b4b5f0-2f6e-4916-8278-ecf886457a6b","Type":"ContainerStarted","Data":"82e331e9e0db6b7dc7bc171ff1013a0269829124c290d598781ab4c331d2f74f"}
Jul 21 04:42:46.309506 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:46.309462    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j" podStartSLOduration=1.309448301 podStartE2EDuration="1.309448301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:42:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:42:46.308832866 +0000 UTC m=+1986.694321799" watchObservedRunningTime="2026-07-21 04:42:46.309448301 +0000 UTC m=+1986.694937209"
Jul 21 04:42:47.301777 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:47.301744    2546 generic.go:356] "Generic (PLEG): container finished" podID="b0b4b5f0-2f6e-4916-8278-ecf886457a6b" containerID="b055fc201c89069b3f3f5d487f1c61026bcc21c1f2327dd9a27b7deb8e5bbeef" exitCode=0
Jul 21 04:42:47.302209 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:47.301800    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j" event={"ID":"b0b4b5f0-2f6e-4916-8278-ecf886457a6b","Type":"ContainerDied","Data":"b055fc201c89069b3f3f5d487f1c61026bcc21c1f2327dd9a27b7deb8e5bbeef"}
Jul 21 04:42:48.443038 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:48.443017    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"
Jul 21 04:42:48.523469 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:48.523430    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0b4b5f0-2f6e-4916-8278-ecf886457a6b-maas-api-service-ca\") pod \"b0b4b5f0-2f6e-4916-8278-ecf886457a6b\" (UID: \"b0b4b5f0-2f6e-4916-8278-ecf886457a6b\") "
Jul 21 04:42:48.523799 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:48.523774    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0b4b5f0-2f6e-4916-8278-ecf886457a6b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b0b4b5f0-2f6e-4916-8278-ecf886457a6b" (UID: "b0b4b5f0-2f6e-4916-8278-ecf886457a6b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:42:48.624695 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:48.624628    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0b4b5f0-2f6e-4916-8278-ecf886457a6b-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:42:49.313635 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:49.313601    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j" event={"ID":"b0b4b5f0-2f6e-4916-8278-ecf886457a6b","Type":"ContainerDied","Data":"82e331e9e0db6b7dc7bc171ff1013a0269829124c290d598781ab4c331d2f74f"}
Jul 21 04:42:49.313635 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:49.313632    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"
Jul 21 04:42:49.313841 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:42:49.313650    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82e331e9e0db6b7dc7bc171ff1013a0269829124c290d598781ab4c331d2f74f"
Jul 21 04:43:00.753759 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:00.753718    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"]
Jul 21 04:43:00.754189 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:00.753921    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl" podUID="ade1de17-bbbe-4c4a-88d9-debec940af4c" containerName="maas-api" containerID="cri-o://0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242" gracePeriod=30
Jul 21 04:43:00.778148 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:00.778088    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:01.010918 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.010863    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:43:01.039079 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.039053    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade1de17-bbbe-4c4a-88d9-debec940af4c-maas-api-tls\") pod \"ade1de17-bbbe-4c4a-88d9-debec940af4c\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") "
Jul 21 04:43:01.039229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.039100    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtm57\" (UniqueName: \"kubernetes.io/projected/ade1de17-bbbe-4c4a-88d9-debec940af4c-kube-api-access-mtm57\") pod \"ade1de17-bbbe-4c4a-88d9-debec940af4c\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") "
Jul 21 04:43:01.039229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.039152    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade1de17-bbbe-4c4a-88d9-debec940af4c-openshift-service-ca\") pod \"ade1de17-bbbe-4c4a-88d9-debec940af4c\" (UID: \"ade1de17-bbbe-4c4a-88d9-debec940af4c\") "
Jul 21 04:43:01.039645 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.039613    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ade1de17-bbbe-4c4a-88d9-debec940af4c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ade1de17-bbbe-4c4a-88d9-debec940af4c" (UID: "ade1de17-bbbe-4c4a-88d9-debec940af4c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:43:01.041396 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.041370    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ade1de17-bbbe-4c4a-88d9-debec940af4c-kube-api-access-mtm57" (OuterVolumeSpecName: "kube-api-access-mtm57") pod "ade1de17-bbbe-4c4a-88d9-debec940af4c" (UID: "ade1de17-bbbe-4c4a-88d9-debec940af4c"). InnerVolumeSpecName "kube-api-access-mtm57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:43:01.041396 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.041386    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ade1de17-bbbe-4c4a-88d9-debec940af4c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ade1de17-bbbe-4c4a-88d9-debec940af4c" (UID: "ade1de17-bbbe-4c4a-88d9-debec940af4c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:43:01.139943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.139901    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtm57\" (UniqueName: \"kubernetes.io/projected/ade1de17-bbbe-4c4a-88d9-debec940af4c-kube-api-access-mtm57\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:43:01.139943 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.139937    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade1de17-bbbe-4c4a-88d9-debec940af4c-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:43:01.140187 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.139953    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade1de17-bbbe-4c4a-88d9-debec940af4c-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:43:01.382080 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.381996    2546 generic.go:356] "Generic (PLEG): container finished" podID="ade1de17-bbbe-4c4a-88d9-debec940af4c" containerID="0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242" exitCode=0
Jul 21 04:43:01.382080 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.382043    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"
Jul 21 04:43:01.382080 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.382069    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl" event={"ID":"ade1de17-bbbe-4c4a-88d9-debec940af4c","Type":"ContainerDied","Data":"0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242"}
Jul 21 04:43:01.382330 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.382141    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl" event={"ID":"ade1de17-bbbe-4c4a-88d9-debec940af4c","Type":"ContainerDied","Data":"5f0f938338941808436f23d14fa1721d23d41a3b3afacd248977b5593fd91bc2"}
Jul 21 04:43:01.382330 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.382175    2546 scope.go:117] "RemoveContainer" containerID="0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242"
Jul 21 04:43:01.391819 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.391794    2546 scope.go:117] "RemoveContainer" containerID="0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242"
Jul 21 04:43:01.392174 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:43:01.392150    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242\": container with ID starting with 0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242 not found: ID does not exist" containerID="0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242"
Jul 21 04:43:01.392247 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.392189    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242"} err="failed to get container status \"0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242\": rpc error: code = NotFound desc = could not find container \"0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242\": container with ID starting with 0a67637d5736003874dfc26f5847dd26187ea5a0e8ee332a74eb029844316242 not found: ID does not exist"
Jul 21 04:43:01.407427 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.407404    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"]
Jul 21 04:43:01.411122 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:01.411090    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6ae222fa-6df5c577bd-7ngnl"]
Jul 21 04:43:02.163368 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:02.163335    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ade1de17-bbbe-4c4a-88d9-debec940af4c" path="/var/lib/kubelet/pods/ade1de17-bbbe-4c4a-88d9-debec940af4c/volumes"
Jul 21 04:43:05.416279 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:05.416245    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"]
Jul 21 04:43:05.419104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:05.419077    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6ae222fa-h9b9j"]
Jul 21 04:43:05.431456 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:05.431426    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:06.163666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:06.163633    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0b4b5f0-2f6e-4916-8278-ecf886457a6b" path="/var/lib/kubelet/pods/b0b4b5f0-2f6e-4916-8278-ecf886457a6b/volumes"
Jul 21 04:43:07.768019 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:07.767980    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:12.789605 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:12.789566    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:40.269762 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:40.269729    2546 scope.go:117] "RemoveContainer" containerID="54e41405bcabf91a8ceacd417485568a13f931f67e174142e0d94a8cc823b046"
Jul 21 04:43:40.277999 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:40.277977    2546 scope.go:117] "RemoveContainer" containerID="1293e63dd7a34e31bd2186e6a25cc4b9b28d14959ebc4683d5674a59898e737d"
Jul 21 04:43:47.104415 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:47.104378    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:49.217636 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:49.217604    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:49.382524 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:49.382492    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:49.760873 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:49.760838    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:49.865183 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:49.865150    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:49.976731 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:49.976697    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:55.652096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:55.652059    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:55.717034 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:55.717004    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:55.940669 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:55.940593    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:56.212369 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:56.212298    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:57.275443 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:57.275408    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:57.368042 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:57.368012    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:57.468078 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:57.468044    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:58.231365 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:58.231323    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:58.323424 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:58.323395    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:58.424813 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:58.424768    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:43:59.843515 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:43:59.843478    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:05.909769 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:05.909736    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:06.039427 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:06.039393    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:06.601515 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:06.601481    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:13.081736 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:13.081693    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:13.182064 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:13.182034    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:23.737001 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:23.736969    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:24.052707 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:24.052606    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:40.218440 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:40.218409    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:44:40.223890 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:40.223872    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:44:40.303678 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:40.303650    2546 scope.go:117] "RemoveContainer" containerID="70fd5c4485252080d539ef2fbec5701de43426f6b2a5ebc06ae7cd4539356e72"
Jul 21 04:44:45.067906 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:45.067868    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:51.197571 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:51.197534    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:51.779397 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:51.779368    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:51.879768 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:51.879739    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:52.131307 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:52.131212    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:52.452695 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:52.452589    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:52.662513 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:52.662477    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:55.040457 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.040426    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:55.117729 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.117697    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:55.117890 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.117778    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"]
Jul 21 04:44:55.118320 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.118304    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0b4b5f0-2f6e-4916-8278-ecf886457a6b" containerName="revoke-keys"
Jul 21 04:44:55.118383 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.118326    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0b4b5f0-2f6e-4916-8278-ecf886457a6b" containerName="revoke-keys"
Jul 21 04:44:55.118383 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.118343    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade1de17-bbbe-4c4a-88d9-debec940af4c" containerName="maas-api"
Jul 21 04:44:55.118383 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.118349    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade1de17-bbbe-4c4a-88d9-debec940af4c" containerName="maas-api"
Jul 21 04:44:55.118482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.118419    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="ade1de17-bbbe-4c4a-88d9-debec940af4c" containerName="maas-api"
Jul 21 04:44:55.118482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.118427    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0b4b5f0-2f6e-4916-8278-ecf886457a6b" containerName="revoke-keys"
Jul 21 04:44:55.121624 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.121608    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.124103 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.124080    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-24ee3b-serving-cert\""
Jul 21 04:44:55.124267 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.124178    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gvc8n\""
Jul 21 04:44:55.131534 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.131513    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"]
Jul 21 04:44:55.174842 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.174802    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27512d1d-5871-43a9-a56c-684bd5773c7d-maas-api-tls\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.175027 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.174852    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmf47\" (UniqueName: \"kubernetes.io/projected/27512d1d-5871-43a9-a56c-684bd5773c7d-kube-api-access-vmf47\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.175027 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.174896    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27512d1d-5871-43a9-a56c-684bd5773c7d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.194630 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.194595    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:55.276523 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.276439    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27512d1d-5871-43a9-a56c-684bd5773c7d-maas-api-tls\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.276776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.276531    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmf47\" (UniqueName: \"kubernetes.io/projected/27512d1d-5871-43a9-a56c-684bd5773c7d-kube-api-access-vmf47\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.276776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.276572    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27512d1d-5871-43a9-a56c-684bd5773c7d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.276776 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:44:55.276576    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-serving-cert: secret "maas-api-e2e-inf-a-24ee3b-serving-cert" not found
Jul 21 04:44:55.276776 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:44:55.276657    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27512d1d-5871-43a9-a56c-684bd5773c7d-maas-api-tls podName:27512d1d-5871-43a9-a56c-684bd5773c7d nodeName:}" failed. No retries permitted until 2026-07-21 04:44:55.776632384 +0000 UTC m=+2116.162121294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/27512d1d-5871-43a9-a56c-684bd5773c7d-maas-api-tls") pod "maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk" (UID: "27512d1d-5871-43a9-a56c-684bd5773c7d") : secret "maas-api-e2e-inf-a-24ee3b-serving-cert" not found
Jul 21 04:44:55.277712 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.277631    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27512d1d-5871-43a9-a56c-684bd5773c7d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.286907 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.286881    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmf47\" (UniqueName: \"kubernetes.io/projected/27512d1d-5871-43a9-a56c-684bd5773c7d-kube-api-access-vmf47\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.345814 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.345730    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:55.716788 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.716758    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:55.781421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.781384    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27512d1d-5871-43a9-a56c-684bd5773c7d-maas-api-tls\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.784035 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.784011    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27512d1d-5871-43a9-a56c-684bd5773c7d-maas-api-tls\") pod \"maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:55.816656 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:55.816623    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:56.035224 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:56.035096    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:56.184164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:56.184140    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"]
Jul 21 04:44:56.185810 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:44:56.185783    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27512d1d_5871_43a9_a56c_684bd5773c7d.slice/crio-706a8411dd3b821a3bceefd143bfa0626a5dbdb574cd405ddd8ec678f3cbb2f3 WatchSource:0}: Error finding container 706a8411dd3b821a3bceefd143bfa0626a5dbdb574cd405ddd8ec678f3cbb2f3: Status 404 returned error can't find the container with id 706a8411dd3b821a3bceefd143bfa0626a5dbdb574cd405ddd8ec678f3cbb2f3
Jul 21 04:44:56.982573 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:56.982538    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk" event={"ID":"27512d1d-5871-43a9-a56c-684bd5773c7d","Type":"ContainerStarted","Data":"f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad"}
Jul 21 04:44:56.982765 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:56.982621    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk" event={"ID":"27512d1d-5871-43a9-a56c-684bd5773c7d","Type":"ContainerStarted","Data":"706a8411dd3b821a3bceefd143bfa0626a5dbdb574cd405ddd8ec678f3cbb2f3"}
Jul 21 04:44:56.982765 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:56.982656    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:44:57.000773 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:57.000717    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk" podStartSLOduration=1.595203232 podStartE2EDuration="2.000702133s" totalImagesPullingTime="405.498901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:44:56.998708707 +0000 UTC m=+2117.384197642" watchObservedRunningTime="2026-07-21 04:44:57.000702133 +0000 UTC m=+2117.386191042"
Jul 21 04:44:59.578134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:59.578070    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:44:59.914909 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:44:59.914832    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:00.130977 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.130937    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"]
Jul 21 04:45:00.134478 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.134453    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n"]
Jul 21 04:45:00.134626 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.134581    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"
Jul 21 04:45:00.137328 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.137292    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:45:00.138291 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.138265    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n"
Jul 21 04:45:00.145634 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.145603    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"]
Jul 21 04:45:00.153605 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.153579    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"
Jul 21 04:45:00.158405 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.158303    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n"]
Jul 21 04:45:00.163371 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.163343    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n"
Jul 21 04:45:00.243908 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.243644    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:00.341226 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.341163    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"]
Jul 21 04:45:00.345825 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:45:00.344771    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575d455a_396d_41b7_b2a3_276be8dd7efc.slice/crio-bc04f9cecba309b356a90ea8768d7a0b5fc3cef2ecd738a8e5eaef8b439ee165 WatchSource:0}: Error finding container bc04f9cecba309b356a90ea8768d7a0b5fc3cef2ecd738a8e5eaef8b439ee165: Status 404 returned error can't find the container with id bc04f9cecba309b356a90ea8768d7a0b5fc3cef2ecd738a8e5eaef8b439ee165
Jul 21 04:45:00.353427 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.353400    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:00.369838 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:00.369812    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n"]
Jul 21 04:45:00.372582 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:45:00.372556    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a67c6d4_e9cc_4dd7_87b6_15c5e239a53c.slice/crio-e86693b44d4e72d14b13417f65f5357456cda85d0dd74ac3bb6ba8c213cfa782 WatchSource:0}: Error finding container e86693b44d4e72d14b13417f65f5357456cda85d0dd74ac3bb6ba8c213cfa782: Status 404 returned error can't find the container with id e86693b44d4e72d14b13417f65f5357456cda85d0dd74ac3bb6ba8c213cfa782
Jul 21 04:45:01.004050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:01.004005    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" event={"ID":"575d455a-396d-41b7-b2a3-276be8dd7efc","Type":"ContainerStarted","Data":"3d2c8c4092ca7455ff79cb732490b18005ca8760ffead9477b42cc236cf46623"}
Jul 21 04:45:01.004521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:01.004069    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" event={"ID":"575d455a-396d-41b7-b2a3-276be8dd7efc","Type":"ContainerStarted","Data":"bc04f9cecba309b356a90ea8768d7a0b5fc3cef2ecd738a8e5eaef8b439ee165"}
Jul 21 04:45:01.005495 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:01.005474    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n" event={"ID":"7a67c6d4-e9cc-4dd7-87b6-15c5e239a53c","Type":"ContainerStarted","Data":"957adc0fcd4e9278b6b20fd2f4bf57f2f636b43bff1c34b04f538d64e311a90b"}
Jul 21 04:45:01.005621 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:01.005511    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n" event={"ID":"7a67c6d4-e9cc-4dd7-87b6-15c5e239a53c","Type":"ContainerStarted","Data":"e86693b44d4e72d14b13417f65f5357456cda85d0dd74ac3bb6ba8c213cfa782"}
Jul 21 04:45:01.019799 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:01.019743    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" podStartSLOduration=1.019726183 podStartE2EDuration="1.019726183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:45:01.017748325 +0000 UTC m=+2121.403237236" watchObservedRunningTime="2026-07-21 04:45:01.019726183 +0000 UTC m=+2121.405215351"
Jul 21 04:45:01.031778 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:01.031728    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n" podStartSLOduration=1.031710139 podStartE2EDuration="1.031710139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:45:01.030868542 +0000 UTC m=+2121.416357453" watchObservedRunningTime="2026-07-21 04:45:01.031710139 +0000 UTC m=+2121.417199050"
Jul 21 04:45:02.012022 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:02.011986    2546 generic.go:356] "Generic (PLEG): container finished" podID="7a67c6d4-e9cc-4dd7-87b6-15c5e239a53c" containerID="957adc0fcd4e9278b6b20fd2f4bf57f2f636b43bff1c34b04f538d64e311a90b" exitCode=0
Jul 21 04:45:02.012492 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:02.012074    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n" event={"ID":"7a67c6d4-e9cc-4dd7-87b6-15c5e239a53c","Type":"ContainerDied","Data":"957adc0fcd4e9278b6b20fd2f4bf57f2f636b43bff1c34b04f538d64e311a90b"}
Jul 21 04:45:02.013602 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:02.013582    2546 generic.go:356] "Generic (PLEG): container finished" podID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerID="3d2c8c4092ca7455ff79cb732490b18005ca8760ffead9477b42cc236cf46623" exitCode=7
Jul 21 04:45:02.013663 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:02.013640    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" event={"ID":"575d455a-396d-41b7-b2a3-276be8dd7efc","Type":"ContainerDied","Data":"3d2c8c4092ca7455ff79cb732490b18005ca8760ffead9477b42cc236cf46623"}
Jul 21 04:45:02.013779 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:02.013765    2546 scope.go:117] "RemoveContainer" containerID="3d2c8c4092ca7455ff79cb732490b18005ca8760ffead9477b42cc236cf46623"
Jul 21 04:45:02.994773 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:02.994703    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:45:03.022563 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.022523    2546 generic.go:356] "Generic (PLEG): container finished" podID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerID="481820a33ea4ee372252f999b397973851fef7c111b401495ba3325f12ec463a" exitCode=7
Jul 21 04:45:03.022986 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.022602    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" event={"ID":"575d455a-396d-41b7-b2a3-276be8dd7efc","Type":"ContainerDied","Data":"481820a33ea4ee372252f999b397973851fef7c111b401495ba3325f12ec463a"}
Jul 21 04:45:03.022986 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.022659    2546 scope.go:117] "RemoveContainer" containerID="3d2c8c4092ca7455ff79cb732490b18005ca8760ffead9477b42cc236cf46623"
Jul 21 04:45:03.022986 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.022831    2546 scope.go:117] "RemoveContainer" containerID="481820a33ea4ee372252f999b397973851fef7c111b401495ba3325f12ec463a"
Jul 21 04:45:03.023509 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:45:03.023155    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh_odh-ai-gateway-infra(575d455a-396d-41b7-b2a3-276be8dd7efc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc"
Jul 21 04:45:03.051107 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.051072    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:03.154498 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.154469    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:03.177584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.177557    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n"
Jul 21 04:45:03.203393 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.203358    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"]
Jul 21 04:45:03.203787 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.203774    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a67c6d4-e9cc-4dd7-87b6-15c5e239a53c" containerName="cleanup"
Jul 21 04:45:03.203831 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.203788    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a67c6d4-e9cc-4dd7-87b6-15c5e239a53c" containerName="cleanup"
Jul 21 04:45:03.203876 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.203867    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a67c6d4-e9cc-4dd7-87b6-15c5e239a53c" containerName="cleanup"
Jul 21 04:45:03.206900 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.206847    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.211574 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.211545    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-97b818-serving-cert\""
Jul 21 04:45:03.226075 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.226025    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"]
Jul 21 04:45:03.267630 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.267544    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:03.360143 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.360081    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/838bff89-9d65-4bf2-8329-7e684828ea34-maas-api-tls\") pod \"maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.360331 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.360169    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h7x4\" (UniqueName: \"kubernetes.io/projected/838bff89-9d65-4bf2-8329-7e684828ea34-kube-api-access-8h7x4\") pod \"maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.360331 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.360202    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/838bff89-9d65-4bf2-8329-7e684828ea34-openshift-service-ca\") pod \"maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.417622 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.417575    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:03.460727 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.460695    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/838bff89-9d65-4bf2-8329-7e684828ea34-maas-api-tls\") pod \"maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.460936 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.460743    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h7x4\" (UniqueName: \"kubernetes.io/projected/838bff89-9d65-4bf2-8329-7e684828ea34-kube-api-access-8h7x4\") pod \"maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.460936 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.460877    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/838bff89-9d65-4bf2-8329-7e684828ea34-openshift-service-ca\") pod \"maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.461529 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.461509    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/838bff89-9d65-4bf2-8329-7e684828ea34-openshift-service-ca\") pod \"maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.463492 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.463470    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/838bff89-9d65-4bf2-8329-7e684828ea34-maas-api-tls\") pod \"maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.469306 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.469280    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h7x4\" (UniqueName: \"kubernetes.io/projected/838bff89-9d65-4bf2-8329-7e684828ea34-kube-api-access-8h7x4\") pod \"maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.519969 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.519865    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:03.668776 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.668746    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"]
Jul 21 04:45:03.670868 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:45:03.670839    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod838bff89_9d65_4bf2_8329_7e684828ea34.slice/crio-ea4c18a6e83f5aa7dc86b66bfb94fc5eb30ac2bdd5908896e3b9d18e06910273 WatchSource:0}: Error finding container ea4c18a6e83f5aa7dc86b66bfb94fc5eb30ac2bdd5908896e3b9d18e06910273: Status 404 returned error can't find the container with id ea4c18a6e83f5aa7dc86b66bfb94fc5eb30ac2bdd5908896e3b9d18e06910273
Jul 21 04:45:03.728044 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.728013    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:03.827899 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:03.827804    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:04.031176 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:04.031139    2546 scope.go:117] "RemoveContainer" containerID="481820a33ea4ee372252f999b397973851fef7c111b401495ba3325f12ec463a"
Jul 21 04:45:04.031673 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:45:04.031439    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh_odh-ai-gateway-infra(575d455a-396d-41b7-b2a3-276be8dd7efc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc"
Jul 21 04:45:04.032244 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:04.032213    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p" event={"ID":"838bff89-9d65-4bf2-8329-7e684828ea34","Type":"ContainerStarted","Data":"ea4c18a6e83f5aa7dc86b66bfb94fc5eb30ac2bdd5908896e3b9d18e06910273"}
Jul 21 04:45:04.033469 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:04.033447    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n" event={"ID":"7a67c6d4-e9cc-4dd7-87b6-15c5e239a53c","Type":"ContainerDied","Data":"e86693b44d4e72d14b13417f65f5357456cda85d0dd74ac3bb6ba8c213cfa782"}
Jul 21 04:45:04.033578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:04.033468    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743485-clf8n"
Jul 21 04:45:04.033578 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:04.033491    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e86693b44d4e72d14b13417f65f5357456cda85d0dd74ac3bb6ba8c213cfa782"
Jul 21 04:45:05.039797 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:05.039763    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p" event={"ID":"838bff89-9d65-4bf2-8329-7e684828ea34","Type":"ContainerStarted","Data":"3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda"}
Jul 21 04:45:05.040225 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:05.039902    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:05.056461 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:05.056406    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p" podStartSLOduration=1.662134193 podStartE2EDuration="2.056389515s" totalImagesPullingTime="394.255322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:45:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:45:05.055445809 +0000 UTC m=+2125.440934719" watchObservedRunningTime="2026-07-21 04:45:05.056389515 +0000 UTC m=+2125.441878423"
Jul 21 04:45:07.596599 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:07.596565    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:11.051352 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:11.051322    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:45:14.345097 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:14.345060    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:15.157521 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:15.157492    2546 scope.go:117] "RemoveContainer" containerID="481820a33ea4ee372252f999b397973851fef7c111b401495ba3325f12ec463a"
Jul 21 04:45:16.097088 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:16.097056    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" event={"ID":"575d455a-396d-41b7-b2a3-276be8dd7efc","Type":"ContainerStarted","Data":"dd24ffe850f76870b2641628d161340b65e114e52962988d7c4f078860e33db3"}
Jul 21 04:45:17.104224 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:17.104191    2546 generic.go:356] "Generic (PLEG): container finished" podID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerID="dd24ffe850f76870b2641628d161340b65e114e52962988d7c4f078860e33db3" exitCode=0
Jul 21 04:45:17.104726 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:17.104231    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" event={"ID":"575d455a-396d-41b7-b2a3-276be8dd7efc","Type":"ContainerDied","Data":"dd24ffe850f76870b2641628d161340b65e114e52962988d7c4f078860e33db3"}
Jul 21 04:45:17.104726 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:17.104280    2546 scope.go:117] "RemoveContainer" containerID="481820a33ea4ee372252f999b397973851fef7c111b401495ba3325f12ec463a"
Jul 21 04:45:17.129621 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:17.129595    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"]
Jul 21 04:45:18.205786 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:18.205757    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:18.254836 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:18.254807    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"
Jul 21 04:45:18.290964 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:18.290930    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:19.118229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:19.118199    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"
Jul 21 04:45:19.118409 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:19.118244    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh" event={"ID":"575d455a-396d-41b7-b2a3-276be8dd7efc","Type":"ContainerDied","Data":"bc04f9cecba309b356a90ea8768d7a0b5fc3cef2ecd738a8e5eaef8b439ee165"}
Jul 21 04:45:19.118409 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:19.118294    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc04f9cecba309b356a90ea8768d7a0b5fc3cef2ecd738a8e5eaef8b439ee165"
Jul 21 04:45:19.158780 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:19.158743    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"]
Jul 21 04:45:19.159008 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:19.158980    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-24ee3b-29743485-2x7lh"]
Jul 21 04:45:19.211134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:19.211079    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:20.163308 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:20.163258    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" path="/var/lib/kubelet/pods/575d455a-396d-41b7-b2a3-276be8dd7efc/volumes"
Jul 21 04:45:25.949937 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:25.949904    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:26.056562 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:26.050260    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:38.575769 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:38.575683    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:45:40.324846 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:45:40.324818    2546 scope.go:117] "RemoveContainer" containerID="38cb3975e302132e840d2d1682b10d67c6061dab2fd8659858237f0a9f26c086"
Jul 21 04:46:10.494535 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:10.494504    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:22.160816 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:22.160787    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:23.093675 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.093645    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:23.205341 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.205310    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"]
Jul 21 04:46:23.205748 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.205716    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerName="cleanup"
Jul 21 04:46:23.205748 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.205727    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerName="cleanup"
Jul 21 04:46:23.205748 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.205745    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerName="cleanup"
Jul 21 04:46:23.205874 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.205752    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerName="cleanup"
Jul 21 04:46:23.205874 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.205766    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerName="cleanup"
Jul 21 04:46:23.205874 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.205771    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerName="cleanup"
Jul 21 04:46:23.205874 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.205828    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerName="cleanup"
Jul 21 04:46:23.205874 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.205835    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerName="cleanup"
Jul 21 04:46:23.209054 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.209037    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"
Jul 21 04:46:23.211431 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.211412    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:46:23.217229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.216777    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"]
Jul 21 04:46:23.331043 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.331006    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv\" (UID: \"6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"
Jul 21 04:46:23.432167 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.432063    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv\" (UID: \"6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"
Jul 21 04:46:23.432716 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.432694    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv\" (UID: \"6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"
Jul 21 04:46:23.521860 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.521823    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"
Jul 21 04:46:23.664091 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.663739    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"]
Jul 21 04:46:23.666900 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:46:23.666869    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd79e56_4ae9_4d43_b261_f47a9bc7c2ec.slice/crio-8bf4b918c59f10d9b9bdba9be025d2be939992a3146f43c1b0e59f98a6e166c7 WatchSource:0}: Error finding container 8bf4b918c59f10d9b9bdba9be025d2be939992a3146f43c1b0e59f98a6e166c7: Status 404 returned error can't find the container with id 8bf4b918c59f10d9b9bdba9be025d2be939992a3146f43c1b0e59f98a6e166c7
Jul 21 04:46:23.698280 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.698254    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:23.799052 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:23.799021    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:24.455616 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:24.455582    2546 generic.go:356] "Generic (PLEG): container finished" podID="6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec" containerID="63a953e6613fdcbff6bd708e2d869c5726b2fd2e1f5b6b4d4997b4cd03d49ddd" exitCode=0
Jul 21 04:46:24.456063 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:24.455643    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv" event={"ID":"6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec","Type":"ContainerDied","Data":"63a953e6613fdcbff6bd708e2d869c5726b2fd2e1f5b6b4d4997b4cd03d49ddd"}
Jul 21 04:46:24.456063 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:24.455691    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv" event={"ID":"6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec","Type":"ContainerStarted","Data":"8bf4b918c59f10d9b9bdba9be025d2be939992a3146f43c1b0e59f98a6e166c7"}
Jul 21 04:46:25.601186 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:25.601158    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"
Jul 21 04:46:25.755066 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:25.755036    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec-maas-api-service-ca\") pod \"6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec\" (UID: \"6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec\") "
Jul 21 04:46:25.755419 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:25.755396    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec" (UID: "6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:46:25.856621 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:25.856585    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:46:26.467231 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:26.467201    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv" event={"ID":"6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec","Type":"ContainerDied","Data":"8bf4b918c59f10d9b9bdba9be025d2be939992a3146f43c1b0e59f98a6e166c7"}
Jul 21 04:46:26.467231 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:26.467219    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"
Jul 21 04:46:26.467437 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:26.467250    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bf4b918c59f10d9b9bdba9be025d2be939992a3146f43c1b0e59f98a6e166c7"
Jul 21 04:46:33.568636 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.568602    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"]
Jul 21 04:46:33.569067 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.568872    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk" podUID="27512d1d-5871-43a9-a56c-684bd5773c7d" containerName="maas-api" containerID="cri-o://f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad" gracePeriod=30
Jul 21 04:46:33.588862 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.588819    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:33.830468 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.830415    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:46:33.935052 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.935017    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27512d1d-5871-43a9-a56c-684bd5773c7d-maas-api-tls\") pod \"27512d1d-5871-43a9-a56c-684bd5773c7d\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") "
Jul 21 04:46:33.935052 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.935059    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27512d1d-5871-43a9-a56c-684bd5773c7d-openshift-service-ca\") pod \"27512d1d-5871-43a9-a56c-684bd5773c7d\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") "
Jul 21 04:46:33.935313 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.935100    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmf47\" (UniqueName: \"kubernetes.io/projected/27512d1d-5871-43a9-a56c-684bd5773c7d-kube-api-access-vmf47\") pod \"27512d1d-5871-43a9-a56c-684bd5773c7d\" (UID: \"27512d1d-5871-43a9-a56c-684bd5773c7d\") "
Jul 21 04:46:33.935490 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.935461    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27512d1d-5871-43a9-a56c-684bd5773c7d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "27512d1d-5871-43a9-a56c-684bd5773c7d" (UID: "27512d1d-5871-43a9-a56c-684bd5773c7d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:46:33.937397 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.937373    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27512d1d-5871-43a9-a56c-684bd5773c7d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "27512d1d-5871-43a9-a56c-684bd5773c7d" (UID: "27512d1d-5871-43a9-a56c-684bd5773c7d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:46:33.937484 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:33.937444    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27512d1d-5871-43a9-a56c-684bd5773c7d-kube-api-access-vmf47" (OuterVolumeSpecName: "kube-api-access-vmf47") pod "27512d1d-5871-43a9-a56c-684bd5773c7d" (UID: "27512d1d-5871-43a9-a56c-684bd5773c7d"). InnerVolumeSpecName "kube-api-access-vmf47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:46:34.035842 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.035807    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27512d1d-5871-43a9-a56c-684bd5773c7d-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:46:34.035842 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.035837    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27512d1d-5871-43a9-a56c-684bd5773c7d-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:46:34.035842 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.035847    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmf47\" (UniqueName: \"kubernetes.io/projected/27512d1d-5871-43a9-a56c-684bd5773c7d-kube-api-access-vmf47\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:46:34.509301 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.509264    2546 generic.go:356] "Generic (PLEG): container finished" podID="27512d1d-5871-43a9-a56c-684bd5773c7d" containerID="f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad" exitCode=0
Jul 21 04:46:34.509301 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.509305    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"
Jul 21 04:46:34.509520 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.509346    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk" event={"ID":"27512d1d-5871-43a9-a56c-684bd5773c7d","Type":"ContainerDied","Data":"f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad"}
Jul 21 04:46:34.509520 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.509395    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk" event={"ID":"27512d1d-5871-43a9-a56c-684bd5773c7d","Type":"ContainerDied","Data":"706a8411dd3b821a3bceefd143bfa0626a5dbdb574cd405ddd8ec678f3cbb2f3"}
Jul 21 04:46:34.509520 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.509416    2546 scope.go:117] "RemoveContainer" containerID="f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad"
Jul 21 04:46:34.518450 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.518426    2546 scope.go:117] "RemoveContainer" containerID="f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad"
Jul 21 04:46:34.518757 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:46:34.518721    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad\": container with ID starting with f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad not found: ID does not exist" containerID="f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad"
Jul 21 04:46:34.518862 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.518750    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad"} err="failed to get container status \"f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad\": rpc error: code = NotFound desc = could not find container \"f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad\": container with ID starting with f4a0450a904683b24056d7032f7c87718818065f09faf080104a46e8221e01ad not found: ID does not exist"
Jul 21 04:46:34.527883 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.527860    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"]
Jul 21 04:46:34.531805 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:34.531782    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-24ee3b-79dd8b4b48-qp5pk"]
Jul 21 04:46:36.162849 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:36.162817    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27512d1d-5871-43a9-a56c-684bd5773c7d" path="/var/lib/kubelet/pods/27512d1d-5871-43a9-a56c-684bd5773c7d/volumes"
Jul 21 04:46:38.271050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:38.271017    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:38.289468 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:38.289428    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"]
Jul 21 04:46:38.293385 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:38.293361    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-24ee3b-x9zbv"]
Jul 21 04:46:38.698860 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:38.698777    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:40.162882 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:40.162853    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec" path="/var/lib/kubelet/pods/6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec/volumes"
Jul 21 04:46:44.837637 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:44.837600    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:44.938469 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:44.938440    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:45.120103 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.120008    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:45.344122 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.344091    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:45.462749 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.462662    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"]
Jul 21 04:46:45.463183 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.463166    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec" containerName="revoke-keys"
Jul 21 04:46:45.463284 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.463186    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec" containerName="revoke-keys"
Jul 21 04:46:45.463284 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.463217    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27512d1d-5871-43a9-a56c-684bd5773c7d" containerName="maas-api"
Jul 21 04:46:45.463284 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.463227    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="27512d1d-5871-43a9-a56c-684bd5773c7d" containerName="maas-api"
Jul 21 04:46:45.463445 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.463346    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="27512d1d-5871-43a9-a56c-684bd5773c7d" containerName="maas-api"
Jul 21 04:46:45.463445 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.463362    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="575d455a-396d-41b7-b2a3-276be8dd7efc" containerName="cleanup"
Jul 21 04:46:45.463445 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.463373    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd79e56-4ae9-4d43-b261-f47a9bc7c2ec" containerName="revoke-keys"
Jul 21 04:46:45.466465 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.466444    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"
Jul 21 04:46:45.468890 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.468857    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:46:45.473572 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.473461    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"]
Jul 21 04:46:45.538068 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.538031    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/799dc520-085f-4562-b2e4-979072ebfcb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-97b818-ghccj\" (UID: \"799dc520-085f-4562-b2e4-979072ebfcb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"
Jul 21 04:46:45.639096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.639057    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/799dc520-085f-4562-b2e4-979072ebfcb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-97b818-ghccj\" (UID: \"799dc520-085f-4562-b2e4-979072ebfcb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"
Jul 21 04:46:45.639722 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.639702    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/799dc520-085f-4562-b2e4-979072ebfcb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-97b818-ghccj\" (UID: \"799dc520-085f-4562-b2e4-979072ebfcb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"
Jul 21 04:46:45.779986 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.779952    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"
Jul 21 04:46:45.918688 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:45.918666    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"]
Jul 21 04:46:45.921161 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:46:45.921135    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod799dc520_085f_4562_b2e4_979072ebfcb4.slice/crio-21a0b6ed0cdad27ab7fd039e89f57a04148373e2516bfbcbccead8d0075e114d WatchSource:0}: Error finding container 21a0b6ed0cdad27ab7fd039e89f57a04148373e2516bfbcbccead8d0075e114d: Status 404 returned error can't find the container with id 21a0b6ed0cdad27ab7fd039e89f57a04148373e2516bfbcbccead8d0075e114d
Jul 21 04:46:46.151609 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:46.151574    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:46.251832 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:46.251804    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:46.575586 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:46.575531    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj" event={"ID":"799dc520-085f-4562-b2e4-979072ebfcb4","Type":"ContainerStarted","Data":"a74d55da97aea486dd3d87716ceb58ba77958817f3ad0e2487350f8cdbab3e0c"}
Jul 21 04:46:46.575789 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:46.575593    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj" event={"ID":"799dc520-085f-4562-b2e4-979072ebfcb4","Type":"ContainerStarted","Data":"21a0b6ed0cdad27ab7fd039e89f57a04148373e2516bfbcbccead8d0075e114d"}
Jul 21 04:46:47.582233 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:47.582195    2546 generic.go:356] "Generic (PLEG): container finished" podID="799dc520-085f-4562-b2e4-979072ebfcb4" containerID="a74d55da97aea486dd3d87716ceb58ba77958817f3ad0e2487350f8cdbab3e0c" exitCode=0
Jul 21 04:46:47.582233 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:47.582232    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj" event={"ID":"799dc520-085f-4562-b2e4-979072ebfcb4","Type":"ContainerDied","Data":"a74d55da97aea486dd3d87716ceb58ba77958817f3ad0e2487350f8cdbab3e0c"}
Jul 21 04:46:48.721534 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:48.721508    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"
Jul 21 04:46:48.770496 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:48.770467    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/799dc520-085f-4562-b2e4-979072ebfcb4-maas-api-service-ca\") pod \"799dc520-085f-4562-b2e4-979072ebfcb4\" (UID: \"799dc520-085f-4562-b2e4-979072ebfcb4\") "
Jul 21 04:46:48.770810 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:48.770790    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/799dc520-085f-4562-b2e4-979072ebfcb4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "799dc520-085f-4562-b2e4-979072ebfcb4" (UID: "799dc520-085f-4562-b2e4-979072ebfcb4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:46:48.872012 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:48.871941    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/799dc520-085f-4562-b2e4-979072ebfcb4-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:46:49.594812 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:49.594779    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"
Jul 21 04:46:49.594993 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:49.594814    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj" event={"ID":"799dc520-085f-4562-b2e4-979072ebfcb4","Type":"ContainerDied","Data":"21a0b6ed0cdad27ab7fd039e89f57a04148373e2516bfbcbccead8d0075e114d"}
Jul 21 04:46:49.594993 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:49.594868    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21a0b6ed0cdad27ab7fd039e89f57a04148373e2516bfbcbccead8d0075e114d"
Jul 21 04:46:55.972839 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:55.972803    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"]
Jul 21 04:46:55.973295 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:55.973046    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p" podUID="838bff89-9d65-4bf2-8329-7e684828ea34" containerName="maas-api" containerID="cri-o://3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda" gracePeriod=30
Jul 21 04:46:55.993594 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:55.993564    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:46:56.046030 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.045987    2546 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p" podUID="838bff89-9d65-4bf2-8329-7e684828ea34" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.53:8443/health\": dial tcp 10.134.0.53:8443: connect: connection refused"
Jul 21 04:46:56.224543 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.224475    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:46:56.337365 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.337333    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/838bff89-9d65-4bf2-8329-7e684828ea34-maas-api-tls\") pod \"838bff89-9d65-4bf2-8329-7e684828ea34\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") "
Jul 21 04:46:56.337551 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.337396    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/838bff89-9d65-4bf2-8329-7e684828ea34-openshift-service-ca\") pod \"838bff89-9d65-4bf2-8329-7e684828ea34\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") "
Jul 21 04:46:56.337551 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.337441    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8h7x4\" (UniqueName: \"kubernetes.io/projected/838bff89-9d65-4bf2-8329-7e684828ea34-kube-api-access-8h7x4\") pod \"838bff89-9d65-4bf2-8329-7e684828ea34\" (UID: \"838bff89-9d65-4bf2-8329-7e684828ea34\") "
Jul 21 04:46:56.337753 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.337729    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/838bff89-9d65-4bf2-8329-7e684828ea34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "838bff89-9d65-4bf2-8329-7e684828ea34" (UID: "838bff89-9d65-4bf2-8329-7e684828ea34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:46:56.339596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.339562    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/838bff89-9d65-4bf2-8329-7e684828ea34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "838bff89-9d65-4bf2-8329-7e684828ea34" (UID: "838bff89-9d65-4bf2-8329-7e684828ea34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:46:56.339715 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.339639    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/838bff89-9d65-4bf2-8329-7e684828ea34-kube-api-access-8h7x4" (OuterVolumeSpecName: "kube-api-access-8h7x4") pod "838bff89-9d65-4bf2-8329-7e684828ea34" (UID: "838bff89-9d65-4bf2-8329-7e684828ea34"). InnerVolumeSpecName "kube-api-access-8h7x4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:46:56.438878 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.438839    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8h7x4\" (UniqueName: \"kubernetes.io/projected/838bff89-9d65-4bf2-8329-7e684828ea34-kube-api-access-8h7x4\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:46:56.438878 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.438871    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/838bff89-9d65-4bf2-8329-7e684828ea34-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:46:56.438878 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.438881    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/838bff89-9d65-4bf2-8329-7e684828ea34-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:46:56.632212 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.632095    2546 generic.go:356] "Generic (PLEG): container finished" podID="838bff89-9d65-4bf2-8329-7e684828ea34" containerID="3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda" exitCode=0
Jul 21 04:46:56.632212 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.632161    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"
Jul 21 04:46:56.632212 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.632168    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p" event={"ID":"838bff89-9d65-4bf2-8329-7e684828ea34","Type":"ContainerDied","Data":"3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda"}
Jul 21 04:46:56.632212 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.632212    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p" event={"ID":"838bff89-9d65-4bf2-8329-7e684828ea34","Type":"ContainerDied","Data":"ea4c18a6e83f5aa7dc86b66bfb94fc5eb30ac2bdd5908896e3b9d18e06910273"}
Jul 21 04:46:56.632539 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.632232    2546 scope.go:117] "RemoveContainer" containerID="3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda"
Jul 21 04:46:56.643031 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.642971    2546 scope.go:117] "RemoveContainer" containerID="3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda"
Jul 21 04:46:56.643380 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:46:56.643350    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda\": container with ID starting with 3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda not found: ID does not exist" containerID="3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda"
Jul 21 04:46:56.643486 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.643392    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda"} err="failed to get container status \"3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda\": rpc error: code = NotFound desc = could not find container \"3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda\": container with ID starting with 3fa12cf02c21f1a217db169a2cd533b956b6d628ed48e7a184f8dac9e4790eda not found: ID does not exist"
Jul 21 04:46:56.656418 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.656388    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"]
Jul 21 04:46:56.660775 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:56.660754    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-97b818-67cfd57b9-m5h6p"]
Jul 21 04:46:58.162937 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:46:58.162902    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="838bff89-9d65-4bf2-8329-7e684828ea34" path="/var/lib/kubelet/pods/838bff89-9d65-4bf2-8329-7e684828ea34/volumes"
Jul 21 04:47:00.538444 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:00.533430    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:00.538444 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:00.536182    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"]
Jul 21 04:47:00.542219 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:00.542190    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97b818-ghccj"]
Jul 21 04:47:00.978179 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:00.978142    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:02.163273 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:02.163241    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="799dc520-085f-4562-b2e4-979072ebfcb4" path="/var/lib/kubelet/pods/799dc520-085f-4562-b2e4-979072ebfcb4/volumes"
Jul 21 04:47:06.807998 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:06.807963    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:06.878599 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:06.878568    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:07.075822 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:07.075744    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:07.545989 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:07.545955    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:07.673568 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:07.673537    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:07.995836 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:07.995800    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:08.419040 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:08.418966    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:10.824063 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.824031    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:10.898134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.898081    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:10.899462 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.899434    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"]
Jul 21 04:47:10.900053 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.900035    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="838bff89-9d65-4bf2-8329-7e684828ea34" containerName="maas-api"
Jul 21 04:47:10.900164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.900054    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="838bff89-9d65-4bf2-8329-7e684828ea34" containerName="maas-api"
Jul 21 04:47:10.900164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.900081    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="799dc520-085f-4562-b2e4-979072ebfcb4" containerName="revoke-keys"
Jul 21 04:47:10.900164 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.900090    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="799dc520-085f-4562-b2e4-979072ebfcb4" containerName="revoke-keys"
Jul 21 04:47:10.900343 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.900204    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="799dc520-085f-4562-b2e4-979072ebfcb4" containerName="revoke-keys"
Jul 21 04:47:10.900343 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.900218    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="838bff89-9d65-4bf2-8329-7e684828ea34" containerName="maas-api"
Jul 21 04:47:10.903621 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.903601    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:10.906004 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.905985    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f63636-serving-cert\""
Jul 21 04:47:10.906132 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.906073    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gvc8n\""
Jul 21 04:47:10.913063 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.913041    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"]
Jul 21 04:47:10.967584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.967548    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:10.975746 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.975721    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f5f07a-9e1d-470a-a957-586255e4106c-maas-api-tls\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:10.975819 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.975766    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwrq4\" (UniqueName: \"kubernetes.io/projected/95f5f07a-9e1d-470a-a957-586255e4106c-kube-api-access-gwrq4\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:10.975819 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:10.975796    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f5f07a-9e1d-470a-a957-586255e4106c-openshift-service-ca\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:11.077193 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.077059    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f5f07a-9e1d-470a-a957-586255e4106c-maas-api-tls\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:11.077193 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.077106    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwrq4\" (UniqueName: \"kubernetes.io/projected/95f5f07a-9e1d-470a-a957-586255e4106c-kube-api-access-gwrq4\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:11.077193 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.077171    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f5f07a-9e1d-470a-a957-586255e4106c-openshift-service-ca\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:11.077488 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:47:11.077233    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-serving-cert: secret "maas-api-e2e-api-a-f63636-serving-cert" not found
Jul 21 04:47:11.077488 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:47:11.077317    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95f5f07a-9e1d-470a-a957-586255e4106c-maas-api-tls podName:95f5f07a-9e1d-470a-a957-586255e4106c nodeName:}" failed. No retries permitted until 2026-07-21 04:47:11.577294058 +0000 UTC m=+2251.962782951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/95f5f07a-9e1d-470a-a957-586255e4106c-maas-api-tls") pod "maas-api-e2e-api-a-f63636-5698bc489b-t8n9h" (UID: "95f5f07a-9e1d-470a-a957-586255e4106c") : secret "maas-api-e2e-api-a-f63636-serving-cert" not found
Jul 21 04:47:11.077840 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.077821    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f5f07a-9e1d-470a-a957-586255e4106c-openshift-service-ca\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:11.085723 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.085705    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwrq4\" (UniqueName: \"kubernetes.io/projected/95f5f07a-9e1d-470a-a957-586255e4106c-kube-api-access-gwrq4\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:11.245397 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.245363    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:11.474661 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.474627    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:11.575666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.575634    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:11.581877 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.581856    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f5f07a-9e1d-470a-a957-586255e4106c-maas-api-tls\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:11.584542 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.584522    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f5f07a-9e1d-470a-a957-586255e4106c-maas-api-tls\") pod \"maas-api-e2e-api-a-f63636-5698bc489b-t8n9h\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:11.816665 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.816565    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:11.961739 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.961716    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"]
Jul 21 04:47:11.963840 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:47:11.963810    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f5f07a_9e1d_470a_a957_586255e4106c.slice/crio-4da06bb354ead692ef89efe716a854fa59febfb624cf15ada0d6765c628cb05e WatchSource:0}: Error finding container 4da06bb354ead692ef89efe716a854fa59febfb624cf15ada0d6765c628cb05e: Status 404 returned error can't find the container with id 4da06bb354ead692ef89efe716a854fa59febfb624cf15ada0d6765c628cb05e
Jul 21 04:47:11.965173 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:11.965156    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 04:47:12.717348 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:12.717317    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h" event={"ID":"95f5f07a-9e1d-470a-a957-586255e4106c","Type":"ContainerStarted","Data":"b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b"}
Jul 21 04:47:12.717657 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:12.717364    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h" event={"ID":"95f5f07a-9e1d-470a-a957-586255e4106c","Type":"ContainerStarted","Data":"4da06bb354ead692ef89efe716a854fa59febfb624cf15ada0d6765c628cb05e"}
Jul 21 04:47:12.717657 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:12.717413    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:12.735104 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:12.735050    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h" podStartSLOduration=2.261386022 podStartE2EDuration="2.735034109s" totalImagesPullingTime="473.648087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:47:12.733341047 +0000 UTC m=+2253.118829976" watchObservedRunningTime="2026-07-21 04:47:12.735034109 +0000 UTC m=+2253.120523017"
Jul 21 04:47:15.345990 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:15.345953    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:15.826779 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:15.826734    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:16.145006 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:16.144923    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:16.503596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:16.503565    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:18.633167 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.633130    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:18.729025 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.728989    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:18.732485 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.732463    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:18.788701 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.788627    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"]
Jul 21 04:47:18.792509 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.792488    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:18.799795 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.799770    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-131f9d-serving-cert\""
Jul 21 04:47:18.805729 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.805702    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"]
Jul 21 04:47:18.838305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.838275    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:18.950933 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.950849    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d5e504-972d-4e64-9d4d-31926e1766fe-openshift-service-ca\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:18.950933 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.950900    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhv9s\" (UniqueName: \"kubernetes.io/projected/49d5e504-972d-4e64-9d4d-31926e1766fe-kube-api-access-zhv9s\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:18.951150 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:18.950945    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d5e504-972d-4e64-9d4d-31926e1766fe-maas-api-tls\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:19.052395 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.052352    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhv9s\" (UniqueName: \"kubernetes.io/projected/49d5e504-972d-4e64-9d4d-31926e1766fe-kube-api-access-zhv9s\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:19.052573 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.052410    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d5e504-972d-4e64-9d4d-31926e1766fe-maas-api-tls\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:19.052573 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.052535    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d5e504-972d-4e64-9d4d-31926e1766fe-openshift-service-ca\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:19.052646 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:47:19.052592    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-serving-cert: secret "maas-api-e2e-api-b-131f9d-serving-cert" not found
Jul 21 04:47:19.052684 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:47:19.052673    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49d5e504-972d-4e64-9d4d-31926e1766fe-maas-api-tls podName:49d5e504-972d-4e64-9d4d-31926e1766fe nodeName:}" failed. No retries permitted until 2026-07-21 04:47:19.552652164 +0000 UTC m=+2259.938141052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49d5e504-972d-4e64-9d4d-31926e1766fe-maas-api-tls") pod "maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt" (UID: "49d5e504-972d-4e64-9d4d-31926e1766fe") : secret "maas-api-e2e-api-b-131f9d-serving-cert" not found
Jul 21 04:47:19.053262 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.053245    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d5e504-972d-4e64-9d4d-31926e1766fe-openshift-service-ca\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:19.063556 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.063533    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhv9s\" (UniqueName: \"kubernetes.io/projected/49d5e504-972d-4e64-9d4d-31926e1766fe-kube-api-access-zhv9s\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:19.248482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.248448    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:19.392991 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.392957    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:19.495771 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.495743    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:19.557624 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.557539    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d5e504-972d-4e64-9d4d-31926e1766fe-maas-api-tls\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:19.560366 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.560336    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d5e504-972d-4e64-9d4d-31926e1766fe-maas-api-tls\") pod \"maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:19.705764 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.705726    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:19.852283 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:19.852223    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"]
Jul 21 04:47:19.854716 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:47:19.854690    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d5e504_972d_4e64_9d4d_31926e1766fe.slice/crio-64f813ab1cea60d2324ff5eb23440eb310fb1c0ea189d50e0f277e08208f34a1 WatchSource:0}: Error finding container 64f813ab1cea60d2324ff5eb23440eb310fb1c0ea189d50e0f277e08208f34a1: Status 404 returned error can't find the container with id 64f813ab1cea60d2324ff5eb23440eb310fb1c0ea189d50e0f277e08208f34a1
Jul 21 04:47:20.767738 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:20.767705    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt" event={"ID":"49d5e504-972d-4e64-9d4d-31926e1766fe","Type":"ContainerStarted","Data":"d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e"}
Jul 21 04:47:20.768150 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:20.767756    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt" event={"ID":"49d5e504-972d-4e64-9d4d-31926e1766fe","Type":"ContainerStarted","Data":"64f813ab1cea60d2324ff5eb23440eb310fb1c0ea189d50e0f277e08208f34a1"}
Jul 21 04:47:20.768150 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:20.767793    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:20.785204 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:20.785160    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt" podStartSLOduration=2.479144259 podStartE2EDuration="2.785145573s" totalImagesPullingTime="306.001314ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:47:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:47:20.783835034 +0000 UTC m=+2261.169323958" watchObservedRunningTime="2026-07-21 04:47:20.785145573 +0000 UTC m=+2261.170634478"
Jul 21 04:47:23.166219 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:23.166190    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:26.778819 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:26.778787    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:47:31.266553 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.266504    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"]
Jul 21 04:47:31.270522 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.270497    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"
Jul 21 04:47:31.272778 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.272758    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:47:31.276616 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.276596    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"]
Jul 21 04:47:31.372321 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.372280    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d7a786c-c3ae-43ec-ab24-5e0b364704ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f63636-l9h4g\" (UID: \"9d7a786c-c3ae-43ec-ab24-5e0b364704ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"
Jul 21 04:47:31.473074 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.473037    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d7a786c-c3ae-43ec-ab24-5e0b364704ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f63636-l9h4g\" (UID: \"9d7a786c-c3ae-43ec-ab24-5e0b364704ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"
Jul 21 04:47:31.473644 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.473624    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d7a786c-c3ae-43ec-ab24-5e0b364704ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f63636-l9h4g\" (UID: \"9d7a786c-c3ae-43ec-ab24-5e0b364704ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"
Jul 21 04:47:31.583671 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.583592    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"
Jul 21 04:47:31.723608 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.723583    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"]
Jul 21 04:47:31.725905 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:47:31.725867    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d7a786c_c3ae_43ec_ab24_5e0b364704ce.slice/crio-d7aafb756e43f46ce276999675448714ad9a3e196bfb3f5263cbcab77bcada07 WatchSource:0}: Error finding container d7aafb756e43f46ce276999675448714ad9a3e196bfb3f5263cbcab77bcada07: Status 404 returned error can't find the container with id d7aafb756e43f46ce276999675448714ad9a3e196bfb3f5263cbcab77bcada07
Jul 21 04:47:31.827385 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.827349    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:31.829581 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.829556    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g" event={"ID":"9d7a786c-c3ae-43ec-ab24-5e0b364704ce","Type":"ContainerStarted","Data":"d7aafb756e43f46ce276999675448714ad9a3e196bfb3f5263cbcab77bcada07"}
Jul 21 04:47:31.930794 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:31.930718    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:32.837636 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:32.837544    2546 generic.go:356] "Generic (PLEG): container finished" podID="9d7a786c-c3ae-43ec-ab24-5e0b364704ce" containerID="cfac51a8b850c6852a8c0bc7c8ab4ecca72427b620b9205905e2e4234bf65edf" exitCode=0
Jul 21 04:47:32.838000 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:32.837631    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g" event={"ID":"9d7a786c-c3ae-43ec-ab24-5e0b364704ce","Type":"ContainerDied","Data":"cfac51a8b850c6852a8c0bc7c8ab4ecca72427b620b9205905e2e4234bf65edf"}
Jul 21 04:47:33.981600 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:33.981572    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"
Jul 21 04:47:34.098031 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:34.097941    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d7a786c-c3ae-43ec-ab24-5e0b364704ce-maas-api-service-ca\") pod \"9d7a786c-c3ae-43ec-ab24-5e0b364704ce\" (UID: \"9d7a786c-c3ae-43ec-ab24-5e0b364704ce\") "
Jul 21 04:47:34.098359 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:34.098335    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d7a786c-c3ae-43ec-ab24-5e0b364704ce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d7a786c-c3ae-43ec-ab24-5e0b364704ce" (UID: "9d7a786c-c3ae-43ec-ab24-5e0b364704ce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:47:34.199138 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:34.199090    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d7a786c-c3ae-43ec-ab24-5e0b364704ce-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:47:34.851421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:34.851380    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g" event={"ID":"9d7a786c-c3ae-43ec-ab24-5e0b364704ce","Type":"ContainerDied","Data":"d7aafb756e43f46ce276999675448714ad9a3e196bfb3f5263cbcab77bcada07"}
Jul 21 04:47:34.851421 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:34.851414    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"
Jul 21 04:47:34.851626 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:34.851454    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7aafb756e43f46ce276999675448714ad9a3e196bfb3f5263cbcab77bcada07"
Jul 21 04:47:40.358572 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:40.358543    2546 scope.go:117] "RemoveContainer" containerID="f31000bb6e4cfa2c1ece9ed6ffcd1e525ecb1d8507f2689fb4a3e4bb99c763ad"
Jul 21 04:47:41.599139 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:41.599085    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:41.701698 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:41.701665    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:46.846875 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:46.846827    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"]
Jul 21 04:47:46.847391 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:46.847146    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h" podUID="95f5f07a-9e1d-470a-a957-586255e4106c" containerName="maas-api" containerID="cri-o://b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b" gracePeriod=30
Jul 21 04:47:46.871861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:46.871834    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:47.105723 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.105663    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:47.227410 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.227375    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f5f07a-9e1d-470a-a957-586255e4106c-openshift-service-ca\") pod \"95f5f07a-9e1d-470a-a957-586255e4106c\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") "
Jul 21 04:47:47.227410 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.227414    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f5f07a-9e1d-470a-a957-586255e4106c-maas-api-tls\") pod \"95f5f07a-9e1d-470a-a957-586255e4106c\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") "
Jul 21 04:47:47.227661 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.227462    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwrq4\" (UniqueName: \"kubernetes.io/projected/95f5f07a-9e1d-470a-a957-586255e4106c-kube-api-access-gwrq4\") pod \"95f5f07a-9e1d-470a-a957-586255e4106c\" (UID: \"95f5f07a-9e1d-470a-a957-586255e4106c\") "
Jul 21 04:47:47.227825 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.227801    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95f5f07a-9e1d-470a-a957-586255e4106c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "95f5f07a-9e1d-470a-a957-586255e4106c" (UID: "95f5f07a-9e1d-470a-a957-586255e4106c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:47:47.229745 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.229715    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95f5f07a-9e1d-470a-a957-586255e4106c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "95f5f07a-9e1d-470a-a957-586255e4106c" (UID: "95f5f07a-9e1d-470a-a957-586255e4106c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:47:47.229745 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.229715    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95f5f07a-9e1d-470a-a957-586255e4106c-kube-api-access-gwrq4" (OuterVolumeSpecName: "kube-api-access-gwrq4") pod "95f5f07a-9e1d-470a-a957-586255e4106c" (UID: "95f5f07a-9e1d-470a-a957-586255e4106c"). InnerVolumeSpecName "kube-api-access-gwrq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:47:47.328948 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.328916    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f5f07a-9e1d-470a-a957-586255e4106c-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:47:47.328948 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.328942    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f5f07a-9e1d-470a-a957-586255e4106c-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:47:47.328948 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.328951    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwrq4\" (UniqueName: \"kubernetes.io/projected/95f5f07a-9e1d-470a-a957-586255e4106c-kube-api-access-gwrq4\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:47:47.923029 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.922998    2546 generic.go:356] "Generic (PLEG): container finished" podID="95f5f07a-9e1d-470a-a957-586255e4106c" containerID="b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b" exitCode=0
Jul 21 04:47:47.923464 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.923043    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h" event={"ID":"95f5f07a-9e1d-470a-a957-586255e4106c","Type":"ContainerDied","Data":"b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b"}
Jul 21 04:47:47.923464 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.923049    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"
Jul 21 04:47:47.923464 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.923086    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h" event={"ID":"95f5f07a-9e1d-470a-a957-586255e4106c","Type":"ContainerDied","Data":"4da06bb354ead692ef89efe716a854fa59febfb624cf15ada0d6765c628cb05e"}
Jul 21 04:47:47.923464 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.923106    2546 scope.go:117] "RemoveContainer" containerID="b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b"
Jul 21 04:47:47.932930 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.932910    2546 scope.go:117] "RemoveContainer" containerID="b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b"
Jul 21 04:47:47.933230 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:47:47.933207    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b\": container with ID starting with b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b not found: ID does not exist" containerID="b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b"
Jul 21 04:47:47.933312 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.933237    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b"} err="failed to get container status \"b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b\": rpc error: code = NotFound desc = could not find container \"b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b\": container with ID starting with b7174dd0164e76a3004b2cf5b0b3522b62f3cecc4677c1f9c0449faf7627b64b not found: ID does not exist"
Jul 21 04:47:47.946067 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.946034    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"]
Jul 21 04:47:47.952660 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:47.952635    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f63636-5698bc489b-t8n9h"]
Jul 21 04:47:48.163202 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:48.163173    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95f5f07a-9e1d-470a-a957-586255e4106c" path="/var/lib/kubelet/pods/95f5f07a-9e1d-470a-a957-586255e4106c/volumes"
Jul 21 04:47:51.347482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:51.347448    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"]
Jul 21 04:47:51.348159 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:51.347821    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:51.350216 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:51.350196    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f63636-l9h4g"]
Jul 21 04:47:51.738539 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:51.738507    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:52.163497 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:52.163419    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d7a786c-c3ae-43ec-ab24-5e0b364704ce" path="/var/lib/kubelet/pods/9d7a786c-c3ae-43ec-ab24-5e0b364704ce/volumes"
Jul 21 04:47:57.782643 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:57.782606    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:57.882899 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:57.882864    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:58.058088 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.058012    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:58.274668 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.274636    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:47:58.403694 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.403618    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"]
Jul 21 04:47:58.404033 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.404021    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95f5f07a-9e1d-470a-a957-586255e4106c" containerName="maas-api"
Jul 21 04:47:58.404075 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.404035    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="95f5f07a-9e1d-470a-a957-586255e4106c" containerName="maas-api"
Jul 21 04:47:58.404075 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.404046    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d7a786c-c3ae-43ec-ab24-5e0b364704ce" containerName="revoke-keys"
Jul 21 04:47:58.404075 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.404051    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d7a786c-c3ae-43ec-ab24-5e0b364704ce" containerName="revoke-keys"
Jul 21 04:47:58.404194 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.404145    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="95f5f07a-9e1d-470a-a957-586255e4106c" containerName="maas-api"
Jul 21 04:47:58.404194 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.404156    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d7a786c-c3ae-43ec-ab24-5e0b364704ce" containerName="revoke-keys"
Jul 21 04:47:58.408383 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.408363    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"
Jul 21 04:47:58.410781 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.410763    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:47:58.414786 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.414762    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"]
Jul 21 04:47:58.436630 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.436603    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-131f9d-z24dw\" (UID: \"72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"
Jul 21 04:47:58.537233 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.537193    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-131f9d-z24dw\" (UID: \"72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"
Jul 21 04:47:58.537830 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.537807    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-131f9d-z24dw\" (UID: \"72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"
Jul 21 04:47:58.721231 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.721199    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"
Jul 21 04:47:58.863003 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.862891    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"]
Jul 21 04:47:58.865607 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:47:58.865571    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72fa84a2_7e92_4db2_a9ed_21df7f2f0bf3.slice/crio-784d973ebebb1166a7ee2fd1916b5cdee5b5b12c4afcfd99201ce5781e347cc7 WatchSource:0}: Error finding container 784d973ebebb1166a7ee2fd1916b5cdee5b5b12c4afcfd99201ce5781e347cc7: Status 404 returned error can't find the container with id 784d973ebebb1166a7ee2fd1916b5cdee5b5b12c4afcfd99201ce5781e347cc7
Jul 21 04:47:58.982948 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:58.982866    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw" event={"ID":"72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3","Type":"ContainerStarted","Data":"784d973ebebb1166a7ee2fd1916b5cdee5b5b12c4afcfd99201ce5781e347cc7"}
Jul 21 04:47:59.990283 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:59.990251    2546 generic.go:356] "Generic (PLEG): container finished" podID="72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3" containerID="c5380688473d40b71949364027a009730f423801d9c697c90b80d15c6bc6fc85" exitCode=0
Jul 21 04:47:59.990655 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:47:59.990337    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw" event={"ID":"72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3","Type":"ContainerDied","Data":"c5380688473d40b71949364027a009730f423801d9c697c90b80d15c6bc6fc85"}
Jul 21 04:48:01.138278 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:01.138254    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"
Jul 21 04:48:01.162438 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:01.162411    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3-maas-api-service-ca\") pod \"72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3\" (UID: \"72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3\") "
Jul 21 04:48:01.162868 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:01.162823    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3" (UID: "72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:48:01.263158 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:01.263030    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:48:02.001632 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:02.001604    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"
Jul 21 04:48:02.001785 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:02.001647    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw" event={"ID":"72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3","Type":"ContainerDied","Data":"784d973ebebb1166a7ee2fd1916b5cdee5b5b12c4afcfd99201ce5781e347cc7"}
Jul 21 04:48:02.001785 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:02.001734    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="784d973ebebb1166a7ee2fd1916b5cdee5b5b12c4afcfd99201ce5781e347cc7"
Jul 21 04:48:13.778955 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:13.778916    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"]
Jul 21 04:48:13.779605 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:13.779220    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt" podUID="49d5e504-972d-4e64-9d4d-31926e1766fe" containerName="maas-api" containerID="cri-o://d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e" gracePeriod=30
Jul 21 04:48:13.808467 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:13.808430    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:14.036481 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.036423    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:48:14.065938 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.065905    2546 generic.go:356] "Generic (PLEG): container finished" podID="49d5e504-972d-4e64-9d4d-31926e1766fe" containerID="d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e" exitCode=0
Jul 21 04:48:14.066101 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.065953    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"
Jul 21 04:48:14.066101 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.065972    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt" event={"ID":"49d5e504-972d-4e64-9d4d-31926e1766fe","Type":"ContainerDied","Data":"d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e"}
Jul 21 04:48:14.066101 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.066024    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt" event={"ID":"49d5e504-972d-4e64-9d4d-31926e1766fe","Type":"ContainerDied","Data":"64f813ab1cea60d2324ff5eb23440eb310fb1c0ea189d50e0f277e08208f34a1"}
Jul 21 04:48:14.066101 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.066056    2546 scope.go:117] "RemoveContainer" containerID="d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e"
Jul 21 04:48:14.076298 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.076276    2546 scope.go:117] "RemoveContainer" containerID="d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e"
Jul 21 04:48:14.076576 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:48:14.076555    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e\": container with ID starting with d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e not found: ID does not exist" containerID="d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e"
Jul 21 04:48:14.076658 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.076584    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e"} err="failed to get container status \"d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e\": rpc error: code = NotFound desc = could not find container \"d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e\": container with ID starting with d24b9c953060a2fe1e601023a59a5080a3bdca4bb00b249c446d22f91e73ce3e not found: ID does not exist"
Jul 21 04:48:14.086927 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.086907    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d5e504-972d-4e64-9d4d-31926e1766fe-maas-api-tls\") pod \"49d5e504-972d-4e64-9d4d-31926e1766fe\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") "
Jul 21 04:48:14.087011 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.086972    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhv9s\" (UniqueName: \"kubernetes.io/projected/49d5e504-972d-4e64-9d4d-31926e1766fe-kube-api-access-zhv9s\") pod \"49d5e504-972d-4e64-9d4d-31926e1766fe\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") "
Jul 21 04:48:14.087050 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.087011    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d5e504-972d-4e64-9d4d-31926e1766fe-openshift-service-ca\") pod \"49d5e504-972d-4e64-9d4d-31926e1766fe\" (UID: \"49d5e504-972d-4e64-9d4d-31926e1766fe\") "
Jul 21 04:48:14.087403 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.087376    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49d5e504-972d-4e64-9d4d-31926e1766fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "49d5e504-972d-4e64-9d4d-31926e1766fe" (UID: "49d5e504-972d-4e64-9d4d-31926e1766fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:48:14.089182 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.089161    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d5e504-972d-4e64-9d4d-31926e1766fe-kube-api-access-zhv9s" (OuterVolumeSpecName: "kube-api-access-zhv9s") pod "49d5e504-972d-4e64-9d4d-31926e1766fe" (UID: "49d5e504-972d-4e64-9d4d-31926e1766fe"). InnerVolumeSpecName "kube-api-access-zhv9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:48:14.089245 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.089182    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49d5e504-972d-4e64-9d4d-31926e1766fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "49d5e504-972d-4e64-9d4d-31926e1766fe" (UID: "49d5e504-972d-4e64-9d4d-31926e1766fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:48:14.188229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.188193    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhv9s\" (UniqueName: \"kubernetes.io/projected/49d5e504-972d-4e64-9d4d-31926e1766fe-kube-api-access-zhv9s\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:48:14.188229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.188219    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d5e504-972d-4e64-9d4d-31926e1766fe-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:48:14.188229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.188229    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d5e504-972d-4e64-9d4d-31926e1766fe-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:48:14.383559 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.383483    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"]
Jul 21 04:48:14.388426 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:14.388400    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-131f9d-798d8dcdc7-5ngqt"]
Jul 21 04:48:16.163271 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:16.163228    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49d5e504-972d-4e64-9d4d-31926e1766fe" path="/var/lib/kubelet/pods/49d5e504-972d-4e64-9d4d-31926e1766fe/volumes"
Jul 21 04:48:18.481520 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:18.481487    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:18.487340 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:18.487315    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"]
Jul 21 04:48:18.491408 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:18.491386    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-131f9d-z24dw"]
Jul 21 04:48:18.896994 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:18.896912    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:20.163907 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:20.163873    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3" path="/var/lib/kubelet/pods/72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3/volumes"
Jul 21 04:48:24.703437 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:24.703387    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:24.804615 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:24.804576    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:24.987415 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:24.987383    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:25.235261 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:25.235218    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:25.575967 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:25.575935    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:25.894383 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:25.894301    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:26.134632 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:26.134596    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:29.281425 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.281393    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:29.374956 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.374920    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:29.375642 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.375619    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"]
Jul 21 04:48:29.376083 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.376065    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d5e504-972d-4e64-9d4d-31926e1766fe" containerName="maas-api"
Jul 21 04:48:29.376203 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.376085    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d5e504-972d-4e64-9d4d-31926e1766fe" containerName="maas-api"
Jul 21 04:48:29.376277 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.376263    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3" containerName="revoke-keys"
Jul 21 04:48:29.376332 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.376280    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3" containerName="revoke-keys"
Jul 21 04:48:29.376403 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.376391    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="72fa84a2-7e92-4db2-a9ed-21df7f2f0bf3" containerName="revoke-keys"
Jul 21 04:48:29.376456 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.376420    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d5e504-972d-4e64-9d4d-31926e1766fe" containerName="maas-api"
Jul 21 04:48:29.381076 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.381055    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.383635 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.383605    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-9fdb31-serving-cert\""
Jul 21 04:48:29.383770 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.383716    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gvc8n\""
Jul 21 04:48:29.390788 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.390761    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"]
Jul 21 04:48:29.463834 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.463803    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:29.537002 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.536909    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbq29\" (UniqueName: \"kubernetes.io/projected/18e6b403-8de6-477d-a30c-57b169b4969b-kube-api-access-kbq29\") pod \"maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.537002 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.536959    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e6b403-8de6-477d-a30c-57b169b4969b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.537270 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.537030    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e6b403-8de6-477d-a30c-57b169b4969b-maas-api-tls\") pod \"maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.637879 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.637842    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbq29\" (UniqueName: \"kubernetes.io/projected/18e6b403-8de6-477d-a30c-57b169b4969b-kube-api-access-kbq29\") pod \"maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.638040 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.637893    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e6b403-8de6-477d-a30c-57b169b4969b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.638040 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.637911    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e6b403-8de6-477d-a30c-57b169b4969b-maas-api-tls\") pod \"maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.638663 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.638638    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e6b403-8de6-477d-a30c-57b169b4969b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.640673 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.640652    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e6b403-8de6-477d-a30c-57b169b4969b-maas-api-tls\") pod \"maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.652000 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.651976    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbq29\" (UniqueName: \"kubernetes.io/projected/18e6b403-8de6-477d-a30c-57b169b4969b-kube-api-access-kbq29\") pod \"maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.669533 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.669499    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:29.696550 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.696506    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:29.842843 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.842819    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"]
Jul 21 04:48:29.844461 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:48:29.844433    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e6b403_8de6_477d_a30c_57b169b4969b.slice/crio-130872ccdf09ac4851d47c8d43ceea532314bdfc8c132fc11d7ff1d5ca74b8e8 WatchSource:0}: Error finding container 130872ccdf09ac4851d47c8d43ceea532314bdfc8c132fc11d7ff1d5ca74b8e8: Status 404 returned error can't find the container with id 130872ccdf09ac4851d47c8d43ceea532314bdfc8c132fc11d7ff1d5ca74b8e8
Jul 21 04:48:29.955522 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:29.955483    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:30.055129 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:30.055092    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:30.171347 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:30.171247    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6" event={"ID":"18e6b403-8de6-477d-a30c-57b169b4969b","Type":"ContainerStarted","Data":"130872ccdf09ac4851d47c8d43ceea532314bdfc8c132fc11d7ff1d5ca74b8e8"}
Jul 21 04:48:31.163697 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:31.163617    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6" event={"ID":"18e6b403-8de6-477d-a30c-57b169b4969b","Type":"ContainerStarted","Data":"52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe"}
Jul 21 04:48:31.164046 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:31.163728    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:31.182885 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:31.182837    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6" podStartSLOduration=1.831697171 podStartE2EDuration="2.182821976s" totalImagesPullingTime="351.124805ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:48:31.181938543 +0000 UTC m=+2331.567427453" watchObservedRunningTime="2026-07-21 04:48:31.182821976 +0000 UTC m=+2331.568310885"
Jul 21 04:48:33.824058 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:33.824022    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:34.159522 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:34.159435    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:34.476833 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:34.476795    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:34.832680 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:34.832598    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:37.174199 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.174171    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:48:37.350063 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.350021    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:37.479781 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.479743    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:37.509076 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.509045    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"]
Jul 21 04:48:37.513239 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.513218    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.516277 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.516253    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-4de716-serving-cert\""
Jul 21 04:48:37.538931 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.538904    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"]
Jul 21 04:48:37.604420 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.604390    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:37.607672 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.607647    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf8hb\" (UniqueName: \"kubernetes.io/projected/9755d5cf-969d-4c98-b92d-4d2c0411c814-kube-api-access-cf8hb\") pod \"maas-api-e2e-shared-b-4de716-559595c7c-fndmb\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.607795 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.607680    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9755d5cf-969d-4c98-b92d-4d2c0411c814-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4de716-559595c7c-fndmb\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.607795 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.607773    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9755d5cf-969d-4c98-b92d-4d2c0411c814-maas-api-tls\") pod \"maas-api-e2e-shared-b-4de716-559595c7c-fndmb\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.708568 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.708531    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9755d5cf-969d-4c98-b92d-4d2c0411c814-maas-api-tls\") pod \"maas-api-e2e-shared-b-4de716-559595c7c-fndmb\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.708737 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.708621    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf8hb\" (UniqueName: \"kubernetes.io/projected/9755d5cf-969d-4c98-b92d-4d2c0411c814-kube-api-access-cf8hb\") pod \"maas-api-e2e-shared-b-4de716-559595c7c-fndmb\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.708737 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.708657    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9755d5cf-969d-4c98-b92d-4d2c0411c814-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4de716-559595c7c-fndmb\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.709319 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.709299    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9755d5cf-969d-4c98-b92d-4d2c0411c814-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4de716-559595c7c-fndmb\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.711060 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.711043    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9755d5cf-969d-4c98-b92d-4d2c0411c814-maas-api-tls\") pod \"maas-api-e2e-shared-b-4de716-559595c7c-fndmb\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.723921 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.723897    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf8hb\" (UniqueName: \"kubernetes.io/projected/9755d5cf-969d-4c98-b92d-4d2c0411c814-kube-api-access-cf8hb\") pod \"maas-api-e2e-shared-b-4de716-559595c7c-fndmb\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.768682 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.768592    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:37.827805 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.827772    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:37.965515 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:37.965491    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"]
Jul 21 04:48:37.967721 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:48:37.967692    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9755d5cf_969d_4c98_b92d_4d2c0411c814.slice/crio-836f16b6c944744fee7be49fe2b56925e94e1658c220b0c3190beb4b30afb48c WatchSource:0}: Error finding container 836f16b6c944744fee7be49fe2b56925e94e1658c220b0c3190beb4b30afb48c: Status 404 returned error can't find the container with id 836f16b6c944744fee7be49fe2b56925e94e1658c220b0c3190beb4b30afb48c
Jul 21 04:48:38.150221 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:38.150139    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:38.202762 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:38.202728    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb" event={"ID":"9755d5cf-969d-4c98-b92d-4d2c0411c814","Type":"ContainerStarted","Data":"836f16b6c944744fee7be49fe2b56925e94e1658c220b0c3190beb4b30afb48c"}
Jul 21 04:48:38.250656 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:38.250617    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:39.210037 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:39.209997    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb" event={"ID":"9755d5cf-969d-4c98-b92d-4d2c0411c814","Type":"ContainerStarted","Data":"aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf"}
Jul 21 04:48:39.210451 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:39.210057    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:39.228634 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:39.228576    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb" podStartSLOduration=1.736611575 podStartE2EDuration="2.228554893s" totalImagesPullingTime="491.943318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:48:39.228139764 +0000 UTC m=+2339.613628676" watchObservedRunningTime="2026-07-21 04:48:39.228554893 +0000 UTC m=+2339.614043803"
Jul 21 04:48:41.884869 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:41.884838    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:45.224756 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:45.224723    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:48:47.943880 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:47.943845    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:48.040256 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:48.040222    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:50.804912 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:50.804875    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:52.050812 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:52.050776    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:59.004328 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:59.004285    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:48:59.103469 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:48:59.103434    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:49:16.008482 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:49:16.008427    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:49:31.411303 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:49:31.411264    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:49:40.251956 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:49:40.251930    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:49:40.259137 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:49:40.259097    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:49:40.401754 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:49:40.401723    2546 scope.go:117] "RemoveContainer" containerID="b055fc201c89069b3f3f5d487f1c61026bcc21c1f2327dd9a27b7deb8e5bbeef"
Jul 21 04:50:41.823175 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:50:41.823138    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:50:52.764313 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:50:52.764281    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:51:02.829681 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:51:02.829645    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:51:11.952334 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:51:11.952296    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:51:12.052765 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:51:12.052731    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:51:19.040960 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:51:19.040927    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:51:39.099780 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:51:39.099695    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:51:40.191217 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:51:40.191183    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:51:40.435268 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:51:40.435234    2546 scope.go:117] "RemoveContainer" containerID="dd24ffe850f76870b2641628d161340b65e114e52962988d7c4f078860e33db3"
Jul 21 04:51:45.469515 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:51:45.469483    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:52:18.427258 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:52:18.427200    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:52:40.461508 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:52:40.461470    2546 scope.go:117] "RemoveContainer" containerID="63a953e6613fdcbff6bd708e2d869c5726b2fd2e1f5b6b4d4997b4cd03d49ddd"
Jul 21 04:53:38.564936 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:53:38.564898    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:53:39.389581 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:53:39.389548    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:53:40.481753 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:53:40.481720    2546 scope.go:117] "RemoveContainer" containerID="a74d55da97aea486dd3d87716ceb58ba77958817f3ad0e2487350f8cdbab3e0c"
Jul 21 04:53:40.491009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:53:40.490990    2546 scope.go:117] "RemoveContainer" containerID="cfac51a8b850c6852a8c0bc7c8ab4ecca72427b620b9205905e2e4234bf65edf"
Jul 21 04:53:48.798441 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:53:48.798317    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:53:48.895800 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:53:48.895769    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:53:51.535427 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:53:51.535396    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:53:51.638336 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:53:51.638304    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:53:59.716731 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:53:59.716694    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:54:01.189707 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:54:01.189665    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:54:02.517053 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:54:02.517017    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:54:30.671021 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:54:30.670922    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:54:40.284354 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:54:40.284325    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:54:40.296067 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:54:40.296042    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp2zl_c7e1e854-9d5b-4763-89f1-9829d53da287/ovn-acl-logging/0.log"
Jul 21 04:54:40.516545 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:54:40.516516    2546 scope.go:117] "RemoveContainer" containerID="c5380688473d40b71949364027a009730f423801d9c697c90b80d15c6bc6fc85"
Jul 21 04:54:55.537423 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:54:55.537388    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:11.586658 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:11.586629    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:12.599253 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:12.599218    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:14.406307 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:14.406273    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:14.506021 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:14.505990    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:32.202704 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.202661    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"]
Jul 21 04:55:32.206226 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.206210    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"
Jul 21 04:55:32.208647 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.208626    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:55:32.212631 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.212604    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"]
Jul 21 04:55:32.238448 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.238408    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d04e77-31ce-4f2c-8488-fd6946297580-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv\" (UID: \"c6d04e77-31ce-4f2c-8488-fd6946297580\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"
Jul 21 04:55:32.339011 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.338963    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d04e77-31ce-4f2c-8488-fd6946297580-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv\" (UID: \"c6d04e77-31ce-4f2c-8488-fd6946297580\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"
Jul 21 04:55:32.339623 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.339603    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d04e77-31ce-4f2c-8488-fd6946297580-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv\" (UID: \"c6d04e77-31ce-4f2c-8488-fd6946297580\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"
Jul 21 04:55:32.518768 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.518731    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"
Jul 21 04:55:32.658072 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.658048    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"]
Jul 21 04:55:32.660300 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:55:32.660270    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d04e77_31ce_4f2c_8488_fd6946297580.slice/crio-1790b469c39020686c53ce81e1d0b05c4d6b787d6aa525cefbb2869a362055d8 WatchSource:0}: Error finding container 1790b469c39020686c53ce81e1d0b05c4d6b787d6aa525cefbb2869a362055d8: Status 404 returned error can't find the container with id 1790b469c39020686c53ce81e1d0b05c4d6b787d6aa525cefbb2869a362055d8
Jul 21 04:55:32.705383 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.705358    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:32.807454 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:32.807383    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:33.398189 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:33.398152    2546 generic.go:356] "Generic (PLEG): container finished" podID="c6d04e77-31ce-4f2c-8488-fd6946297580" containerID="65f5ac33bd81bd5a157e9ad59cc57d5b25570d00937789db1ac4795c229651e2" exitCode=0
Jul 21 04:55:33.398611 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:33.398210    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv" event={"ID":"c6d04e77-31ce-4f2c-8488-fd6946297580","Type":"ContainerDied","Data":"65f5ac33bd81bd5a157e9ad59cc57d5b25570d00937789db1ac4795c229651e2"}
Jul 21 04:55:33.398611 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:33.398249    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv" event={"ID":"c6d04e77-31ce-4f2c-8488-fd6946297580","Type":"ContainerStarted","Data":"1790b469c39020686c53ce81e1d0b05c4d6b787d6aa525cefbb2869a362055d8"}
Jul 21 04:55:34.546360 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:34.546336    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"
Jul 21 04:55:34.662831 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:34.662741    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d04e77-31ce-4f2c-8488-fd6946297580-maas-api-service-ca\") pod \"c6d04e77-31ce-4f2c-8488-fd6946297580\" (UID: \"c6d04e77-31ce-4f2c-8488-fd6946297580\") "
Jul 21 04:55:34.663175 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:34.663152    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d04e77-31ce-4f2c-8488-fd6946297580-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c6d04e77-31ce-4f2c-8488-fd6946297580" (UID: "c6d04e77-31ce-4f2c-8488-fd6946297580"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:55:34.763842 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:34.763802    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d04e77-31ce-4f2c-8488-fd6946297580-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:55:35.410255 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:35.410226    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"
Jul 21 04:55:35.410439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:35.410264    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv" event={"ID":"c6d04e77-31ce-4f2c-8488-fd6946297580","Type":"ContainerDied","Data":"1790b469c39020686c53ce81e1d0b05c4d6b787d6aa525cefbb2869a362055d8"}
Jul 21 04:55:35.410439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:35.410312    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1790b469c39020686c53ce81e1d0b05c4d6b787d6aa525cefbb2869a362055d8"
Jul 21 04:55:42.596293 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.596246    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"]
Jul 21 04:55:42.596861 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.596534    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6" podUID="18e6b403-8de6-477d-a30c-57b169b4969b" containerName="maas-api" containerID="cri-o://52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe" gracePeriod=30
Jul 21 04:55:42.622222 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.622191    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:42.849188 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.849134    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:55:42.946400 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.946366    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e6b403-8de6-477d-a30c-57b169b4969b-maas-api-tls\") pod \"18e6b403-8de6-477d-a30c-57b169b4969b\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") "
Jul 21 04:55:42.946584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.946414    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e6b403-8de6-477d-a30c-57b169b4969b-openshift-service-ca\") pod \"18e6b403-8de6-477d-a30c-57b169b4969b\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") "
Jul 21 04:55:42.946584 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.946494    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbq29\" (UniqueName: \"kubernetes.io/projected/18e6b403-8de6-477d-a30c-57b169b4969b-kube-api-access-kbq29\") pod \"18e6b403-8de6-477d-a30c-57b169b4969b\" (UID: \"18e6b403-8de6-477d-a30c-57b169b4969b\") "
Jul 21 04:55:42.946856 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.946830    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e6b403-8de6-477d-a30c-57b169b4969b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18e6b403-8de6-477d-a30c-57b169b4969b" (UID: "18e6b403-8de6-477d-a30c-57b169b4969b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:55:42.948692 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.948667    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e6b403-8de6-477d-a30c-57b169b4969b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18e6b403-8de6-477d-a30c-57b169b4969b" (UID: "18e6b403-8de6-477d-a30c-57b169b4969b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:55:42.948775 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:42.948695    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e6b403-8de6-477d-a30c-57b169b4969b-kube-api-access-kbq29" (OuterVolumeSpecName: "kube-api-access-kbq29") pod "18e6b403-8de6-477d-a30c-57b169b4969b" (UID: "18e6b403-8de6-477d-a30c-57b169b4969b"). InnerVolumeSpecName "kube-api-access-kbq29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:55:43.048012 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.047963    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e6b403-8de6-477d-a30c-57b169b4969b-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:55:43.048012 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.048009    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e6b403-8de6-477d-a30c-57b169b4969b-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:55:43.048012 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.048024    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbq29\" (UniqueName: \"kubernetes.io/projected/18e6b403-8de6-477d-a30c-57b169b4969b-kube-api-access-kbq29\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:55:43.451614 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.451581    2546 generic.go:356] "Generic (PLEG): container finished" podID="18e6b403-8de6-477d-a30c-57b169b4969b" containerID="52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe" exitCode=0
Jul 21 04:55:43.451781 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.451621    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6" event={"ID":"18e6b403-8de6-477d-a30c-57b169b4969b","Type":"ContainerDied","Data":"52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe"}
Jul 21 04:55:43.451781 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.451633    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"
Jul 21 04:55:43.451781 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.451668    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6" event={"ID":"18e6b403-8de6-477d-a30c-57b169b4969b","Type":"ContainerDied","Data":"130872ccdf09ac4851d47c8d43ceea532314bdfc8c132fc11d7ff1d5ca74b8e8"}
Jul 21 04:55:43.451781 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.451690    2546 scope.go:117] "RemoveContainer" containerID="52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe"
Jul 21 04:55:43.461269 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.461252    2546 scope.go:117] "RemoveContainer" containerID="52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe"
Jul 21 04:55:43.461527 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:55:43.461510    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe\": container with ID starting with 52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe not found: ID does not exist" containerID="52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe"
Jul 21 04:55:43.461586 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.461538    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe"} err="failed to get container status \"52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe\": rpc error: code = NotFound desc = could not find container \"52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe\": container with ID starting with 52f0f1a9ed4448ce4a47156959c9a5da89f76ff4053ce25577b1ee339cb571fe not found: ID does not exist"
Jul 21 04:55:43.473416 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.473383    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"]
Jul 21 04:55:43.477020 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:43.477002    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9fdb31-578b5cd844-6b6s6"]
Jul 21 04:55:44.163587 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:44.163550    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e6b403-8de6-477d-a30c-57b169b4969b" path="/var/lib/kubelet/pods/18e6b403-8de6-477d-a30c-57b169b4969b/volumes"
Jul 21 04:55:47.278009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:47.277972    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:47.288750 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:47.288720    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"]
Jul 21 04:55:47.292938 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:47.292914    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9fdb31-gbcdv"]
Jul 21 04:55:47.679298 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:47.679211    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:48.162553 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:48.162520    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d04e77-31ce-4f2c-8488-fd6946297580" path="/var/lib/kubelet/pods/c6d04e77-31ce-4f2c-8488-fd6946297580/volumes"
Jul 21 04:55:53.773479 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:53.773441    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:53.874052 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:53.874016    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:54.047625 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.047544    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:54.292344 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.292314    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:54.418486 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.418405    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"]
Jul 21 04:55:54.419013 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.418993    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d04e77-31ce-4f2c-8488-fd6946297580" containerName="revoke-keys"
Jul 21 04:55:54.419061 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.419019    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d04e77-31ce-4f2c-8488-fd6946297580" containerName="revoke-keys"
Jul 21 04:55:54.419095 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.419060    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e6b403-8de6-477d-a30c-57b169b4969b" containerName="maas-api"
Jul 21 04:55:54.419095 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.419070    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e6b403-8de6-477d-a30c-57b169b4969b" containerName="maas-api"
Jul 21 04:55:54.419203 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.419190    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d04e77-31ce-4f2c-8488-fd6946297580" containerName="revoke-keys"
Jul 21 04:55:54.419241 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.419207    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e6b403-8de6-477d-a30c-57b169b4969b" containerName="maas-api"
Jul 21 04:55:54.422834 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.422820    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"
Jul 21 04:55:54.425149 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.425066    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7c5dw\""
Jul 21 04:55:54.428871 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.428841    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"]
Jul 21 04:55:54.553864 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.553819    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa66877-2922-4c47-bda6-dd57042ab3f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg\" (UID: \"efa66877-2922-4c47-bda6-dd57042ab3f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"
Jul 21 04:55:54.654854 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.654823    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa66877-2922-4c47-bda6-dd57042ab3f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg\" (UID: \"efa66877-2922-4c47-bda6-dd57042ab3f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"
Jul 21 04:55:54.655476 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.655458    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa66877-2922-4c47-bda6-dd57042ab3f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg\" (UID: \"efa66877-2922-4c47-bda6-dd57042ab3f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"
Jul 21 04:55:54.736539 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.736513    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"
Jul 21 04:55:54.873438 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:54.873413    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"]
Jul 21 04:55:54.875583 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:55:54.875560    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa66877_2922_4c47_bda6_dd57042ab3f0.slice/crio-50ad95a0b65dffea9d7073deb3770b8bc4b508fc606fe7dea489d1005ab2985a WatchSource:0}: Error finding container 50ad95a0b65dffea9d7073deb3770b8bc4b508fc606fe7dea489d1005ab2985a: Status 404 returned error can't find the container with id 50ad95a0b65dffea9d7073deb3770b8bc4b508fc606fe7dea489d1005ab2985a
Jul 21 04:55:55.106498 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:55.106433    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:55.206924 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:55.206887    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:55.515009 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:55.514974    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg" event={"ID":"efa66877-2922-4c47-bda6-dd57042ab3f0","Type":"ContainerStarted","Data":"a7b78ecde396ec05e0193576cc751f2e69c7a10cfed157bee86240b36dffd44d"}
Jul 21 04:55:55.515220 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:55.515030    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg" event={"ID":"efa66877-2922-4c47-bda6-dd57042ab3f0","Type":"ContainerStarted","Data":"50ad95a0b65dffea9d7073deb3770b8bc4b508fc606fe7dea489d1005ab2985a"}
Jul 21 04:55:55.532206 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:55.532161    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg" podStartSLOduration=1.532144819 podStartE2EDuration="1.532144819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 04:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:55:55.53045664 +0000 UTC m=+2775.915945547" watchObservedRunningTime="2026-07-21 04:55:55.532144819 +0000 UTC m=+2775.917633726"
Jul 21 04:55:56.521510 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:56.521471    2546 generic.go:356] "Generic (PLEG): container finished" podID="efa66877-2922-4c47-bda6-dd57042ab3f0" containerID="a7b78ecde396ec05e0193576cc751f2e69c7a10cfed157bee86240b36dffd44d" exitCode=0
Jul 21 04:55:56.521901 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:56.521536    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg" event={"ID":"efa66877-2922-4c47-bda6-dd57042ab3f0","Type":"ContainerDied","Data":"a7b78ecde396ec05e0193576cc751f2e69c7a10cfed157bee86240b36dffd44d"}
Jul 21 04:55:57.670611 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:57.670590    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"
Jul 21 04:55:57.783286 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:57.783195    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa66877-2922-4c47-bda6-dd57042ab3f0-maas-api-service-ca\") pod \"efa66877-2922-4c47-bda6-dd57042ab3f0\" (UID: \"efa66877-2922-4c47-bda6-dd57042ab3f0\") "
Jul 21 04:55:57.783545 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:57.783524    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa66877-2922-4c47-bda6-dd57042ab3f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "efa66877-2922-4c47-bda6-dd57042ab3f0" (UID: "efa66877-2922-4c47-bda6-dd57042ab3f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:55:57.884867 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:57.884834    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa66877-2922-4c47-bda6-dd57042ab3f0-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:55:58.533790 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:58.533755    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg" event={"ID":"efa66877-2922-4c47-bda6-dd57042ab3f0","Type":"ContainerDied","Data":"50ad95a0b65dffea9d7073deb3770b8bc4b508fc606fe7dea489d1005ab2985a"}
Jul 21 04:55:58.533790 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:58.533777    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"
Jul 21 04:55:58.533971 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:58.533835    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50ad95a0b65dffea9d7073deb3770b8bc4b508fc606fe7dea489d1005ab2985a"
Jul 21 04:55:59.540673 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:59.540640    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:55:59.595063 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:55:59.595035    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:04.808849 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:04.808309    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"]
Jul 21 04:56:04.808849 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:04.808570    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb" podUID="9755d5cf-969d-4c98-b92d-4d2c0411c814" containerName="maas-api" containerID="cri-o://aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf" gracePeriod=30
Jul 21 04:56:04.840364 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:04.840327    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:05.266712 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.266691    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:56:05.350625 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.350557    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9755d5cf-969d-4c98-b92d-4d2c0411c814-maas-api-tls\") pod \"9755d5cf-969d-4c98-b92d-4d2c0411c814\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") "
Jul 21 04:56:05.350758 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.350673    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf8hb\" (UniqueName: \"kubernetes.io/projected/9755d5cf-969d-4c98-b92d-4d2c0411c814-kube-api-access-cf8hb\") pod \"9755d5cf-969d-4c98-b92d-4d2c0411c814\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") "
Jul 21 04:56:05.350758 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.350711    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9755d5cf-969d-4c98-b92d-4d2c0411c814-openshift-service-ca\") pod \"9755d5cf-969d-4c98-b92d-4d2c0411c814\" (UID: \"9755d5cf-969d-4c98-b92d-4d2c0411c814\") "
Jul 21 04:56:05.351069 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.351043    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9755d5cf-969d-4c98-b92d-4d2c0411c814-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9755d5cf-969d-4c98-b92d-4d2c0411c814" (UID: "9755d5cf-969d-4c98-b92d-4d2c0411c814"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 04:56:05.352845 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.352822    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9755d5cf-969d-4c98-b92d-4d2c0411c814-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9755d5cf-969d-4c98-b92d-4d2c0411c814" (UID: "9755d5cf-969d-4c98-b92d-4d2c0411c814"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 04:56:05.352967 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.352944    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9755d5cf-969d-4c98-b92d-4d2c0411c814-kube-api-access-cf8hb" (OuterVolumeSpecName: "kube-api-access-cf8hb") pod "9755d5cf-969d-4c98-b92d-4d2c0411c814" (UID: "9755d5cf-969d-4c98-b92d-4d2c0411c814"). InnerVolumeSpecName "kube-api-access-cf8hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 04:56:05.451547 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.451517    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9755d5cf-969d-4c98-b92d-4d2c0411c814-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:56:05.451547 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.451542    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9755d5cf-969d-4c98-b92d-4d2c0411c814-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:56:05.451547 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.451551    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf8hb\" (UniqueName: \"kubernetes.io/projected/9755d5cf-969d-4c98-b92d-4d2c0411c814-kube-api-access-cf8hb\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Jul 21 04:56:05.570576 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.570545    2546 generic.go:356] "Generic (PLEG): container finished" podID="9755d5cf-969d-4c98-b92d-4d2c0411c814" containerID="aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf" exitCode=0
Jul 21 04:56:05.570750 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.570596    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"
Jul 21 04:56:05.570750 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.570601    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb" event={"ID":"9755d5cf-969d-4c98-b92d-4d2c0411c814","Type":"ContainerDied","Data":"aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf"}
Jul 21 04:56:05.570750 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.570642    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb" event={"ID":"9755d5cf-969d-4c98-b92d-4d2c0411c814","Type":"ContainerDied","Data":"836f16b6c944744fee7be49fe2b56925e94e1658c220b0c3190beb4b30afb48c"}
Jul 21 04:56:05.570750 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.570664    2546 scope.go:117] "RemoveContainer" containerID="aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf"
Jul 21 04:56:05.580011 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.579994    2546 scope.go:117] "RemoveContainer" containerID="aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf"
Jul 21 04:56:05.580280 ip-10-0-140-142 kubenswrapper[2546]: E0721 04:56:05.580263    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf\": container with ID starting with aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf not found: ID does not exist" containerID="aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf"
Jul 21 04:56:05.580338 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.580290    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf"} err="failed to get container status \"aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf\": rpc error: code = NotFound desc = could not find container \"aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf\": container with ID starting with aca602b73a6ee51d4e67a13a4de21b422993a7a6b5e63a12d02ec12dbae51fbf not found: ID does not exist"
Jul 21 04:56:05.595721 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.593954    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"]
Jul 21 04:56:05.602820 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:05.602756    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb"]
Jul 21 04:56:06.163206 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:06.163168    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9755d5cf-969d-4c98-b92d-4d2c0411c814" path="/var/lib/kubelet/pods/9755d5cf-969d-4c98-b92d-4d2c0411c814/volumes"
Jul 21 04:56:08.220484 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:08.220430    2546 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4de716-559595c7c-fndmb" podUID="9755d5cf-969d-4c98-b92d-4d2c0411c814" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.61:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 21 04:56:09.489653 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:09.489622    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"]
Jul 21 04:56:09.490025 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:09.489815    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:09.492587 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:09.492563    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4de716-xdzkg"]
Jul 21 04:56:09.899371 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:09.899296    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:10.162922 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:10.162843    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa66877-2922-4c47-bda6-dd57042ab3f0" path="/var/lib/kubelet/pods/efa66877-2922-4c47-bda6-dd57042ab3f0/volumes"
Jul 21 04:56:16.082308 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:16.082279    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:16.182453 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:16.182421    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:16.390466 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:16.390375    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:16.619496 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:16.619439    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:22.960495 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:22.960462    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6ffdf96fc7-dknzk_b07b3a5d-dde8-462d-b3ed-02f20b685664/manager/0.log"
Jul 21 04:56:23.316787 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:23.316760    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d877cbcc8-5br7f_f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4/copy-manifests/0.log"
Jul 21 04:56:23.329876 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:23.329852    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d877cbcc8-5br7f_f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4/manager/0.log"
Jul 21 04:56:23.941268 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:23.941236    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8666b4847-p2jw4_e0b0a511-8015-44ae-b26f-b1c100980544/maas-api/0.log"
Jul 21 04:56:24.051919 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:24.051884    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743470-xjk25_ee6548d8-08a4-4988-9046-fad547dbffbb/cleanup/0.log"
Jul 21 04:56:24.165385 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:24.165354    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743485-clf8n_7a67c6d4-e9cc-4dd7-87b6-15c5e239a53c/cleanup/0.log"
Jul 21 04:56:26.538548 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:26.538525    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-l9cxb_c1d9a61c-9e77-4203-b226-54e43c763fd6/manager/0.log"
Jul 21 04:56:27.040424 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:27.040401    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hv9vk_791657e0-9d92-4cbc-8804-be0d29845b9a/discovery/0.log"
Jul 21 04:56:27.508562 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:27.508536    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-bgjkd_df0c9777-01ce-4a56-bb96-d5431297daa3/payload-pre-processing/0.log"
Jul 21 04:56:27.634673 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:27.634646    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-4fdww_33c1fcd2-e914-4377-854c-7d2459113f21/payload-processing/0.log"
Jul 21 04:56:28.222961 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:28.222927    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-94tlm_2caf16b1-1d3d-443d-bcbf-835b39fda32a/main/0.log"
Jul 21 04:56:34.802026 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:34.801991    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:34.899766 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:34.899736    2546 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log" limit=1024
Jul 21 04:56:37.472723 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:37.472682    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-75ljs_4f4d15b6-a854-4feb-82a8-3fc155affffb/global-pull-secret-syncer/0.log"
Jul 21 04:56:37.601101 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:37.601072    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-glqxn_30699fbf-ab92-43f7-8d71-d4f04fde4381/konnectivity-agent/0.log"
Jul 21 04:56:37.735096 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:37.735000    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-142.ec2.internal_f5f2842cd541864d0773b17e6d2e6e62/haproxy/0.log"
Jul 21 04:56:43.119062 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:43.118973    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-l9cxb_c1d9a61c-9e77-4203-b226-54e43c763fd6/manager/0.log"
Jul 21 04:56:44.767255 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:44.767229    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_24552c9b-7be4-4e35-a861-ae6b668cd04e/alertmanager/0.log"
Jul 21 04:56:44.789416 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:44.789399    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_24552c9b-7be4-4e35-a861-ae6b668cd04e/config-reloader/0.log"
Jul 21 04:56:44.811228 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:44.811210    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_24552c9b-7be4-4e35-a861-ae6b668cd04e/kube-rbac-proxy-web/0.log"
Jul 21 04:56:44.842046 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:44.842023    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_24552c9b-7be4-4e35-a861-ae6b668cd04e/kube-rbac-proxy/0.log"
Jul 21 04:56:44.865002 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:44.864979    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_24552c9b-7be4-4e35-a861-ae6b668cd04e/kube-rbac-proxy-metric/0.log"
Jul 21 04:56:44.884917 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:44.884897    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_24552c9b-7be4-4e35-a861-ae6b668cd04e/prom-label-proxy/0.log"
Jul 21 04:56:44.907280 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:44.907254    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_24552c9b-7be4-4e35-a861-ae6b668cd04e/init-config-reloader/0.log"
Jul 21 04:56:45.000066 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.000040    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-zd2lp_966786ab-2d3c-49fa-b002-38d13d6eb388/kube-state-metrics/0.log"
Jul 21 04:56:45.022260 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.022201    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-zd2lp_966786ab-2d3c-49fa-b002-38d13d6eb388/kube-rbac-proxy-main/0.log"
Jul 21 04:56:45.048829 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.048810    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-zd2lp_966786ab-2d3c-49fa-b002-38d13d6eb388/kube-rbac-proxy-self/0.log"
Jul 21 04:56:45.075504 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.075481    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f44576f59-v5zpb_f0e709af-3e64-43c7-a199-552fac76dc2d/metrics-server/0.log"
Jul 21 04:56:45.102064 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.102038    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-pfpwp_c7a345e9-5838-4a63-9905-027db812c6af/monitoring-plugin/0.log"
Jul 21 04:56:45.228249 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.228218    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kl7wx_63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf/node-exporter/0.log"
Jul 21 04:56:45.254539 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.254508    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kl7wx_63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf/kube-rbac-proxy/0.log"
Jul 21 04:56:45.278720 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.278660    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kl7wx_63b58ae4-5ef0-4f79-bb55-1cf8dc55f0cf/init-textfile/0.log"
Jul 21 04:56:45.386976 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.386934    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-68cnf_878373ef-ec3e-44fe-8f96-57d91d72435e/kube-rbac-proxy-main/0.log"
Jul 21 04:56:45.409046 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.409018    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-68cnf_878373ef-ec3e-44fe-8f96-57d91d72435e/kube-rbac-proxy-self/0.log"
Jul 21 04:56:45.431551 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.431528    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-68cnf_878373ef-ec3e-44fe-8f96-57d91d72435e/openshift-state-metrics/0.log"
Jul 21 04:56:45.483780 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.483753    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c43ef906-877b-44ef-bc7b-6ecaaafb6da8/prometheus/0.log"
Jul 21 04:56:45.505013 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.504993    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c43ef906-877b-44ef-bc7b-6ecaaafb6da8/config-reloader/0.log"
Jul 21 04:56:45.526524 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.526505    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c43ef906-877b-44ef-bc7b-6ecaaafb6da8/thanos-sidecar/0.log"
Jul 21 04:56:45.554016 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.553940    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c43ef906-877b-44ef-bc7b-6ecaaafb6da8/kube-rbac-proxy-web/0.log"
Jul 21 04:56:45.574072 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.574054    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c43ef906-877b-44ef-bc7b-6ecaaafb6da8/kube-rbac-proxy/0.log"
Jul 21 04:56:45.594666 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.594652    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c43ef906-877b-44ef-bc7b-6ecaaafb6da8/kube-rbac-proxy-thanos/0.log"
Jul 21 04:56:45.615731 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.615718    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c43ef906-877b-44ef-bc7b-6ecaaafb6da8/init-config-reloader/0.log"
Jul 21 04:56:45.642150 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.642129    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-45jck_081284ef-343b-4f90-9981-f23f723f5684/prometheus-operator/0.log"
Jul 21 04:56:45.660738 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.660719    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-45jck_081284ef-343b-4f90-9981-f23f723f5684/kube-rbac-proxy/0.log"
Jul 21 04:56:45.684189 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.684166    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-pmzjh_b968b540-a6d6-4a3c-a848-703d14667630/prometheus-operator-admission-webhook/0.log"
Jul 21 04:56:45.712382 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.712348    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8b5f7967d-nsljl_792fd228-fa50-4842-81ed-727eb3f9639d/telemeter-client/0.log"
Jul 21 04:56:45.730082 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.730059    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8b5f7967d-nsljl_792fd228-fa50-4842-81ed-727eb3f9639d/reload/0.log"
Jul 21 04:56:45.751110 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.751093    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8b5f7967d-nsljl_792fd228-fa50-4842-81ed-727eb3f9639d/kube-rbac-proxy/0.log"
Jul 21 04:56:45.777391 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.777370    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5884f7ddfb-twz9t_b1f82002-4d5f-41b0-89dd-7673c1df3111/thanos-query/0.log"
Jul 21 04:56:45.797183 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.797162    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5884f7ddfb-twz9t_b1f82002-4d5f-41b0-89dd-7673c1df3111/kube-rbac-proxy-web/0.log"
Jul 21 04:56:45.822268 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.822211    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5884f7ddfb-twz9t_b1f82002-4d5f-41b0-89dd-7673c1df3111/kube-rbac-proxy/0.log"
Jul 21 04:56:45.843919 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.843899    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5884f7ddfb-twz9t_b1f82002-4d5f-41b0-89dd-7673c1df3111/prom-label-proxy/0.log"
Jul 21 04:56:45.864987 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.864967    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5884f7ddfb-twz9t_b1f82002-4d5f-41b0-89dd-7673c1df3111/kube-rbac-proxy-rules/0.log"
Jul 21 04:56:45.901133 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:45.901095    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5884f7ddfb-twz9t_b1f82002-4d5f-41b0-89dd-7673c1df3111/kube-rbac-proxy-metrics/0.log"
Jul 21 04:56:47.145942 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.145913    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"]
Jul 21 04:56:47.146596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.146541    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa66877-2922-4c47-bda6-dd57042ab3f0" containerName="revoke-keys"
Jul 21 04:56:47.146596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.146559    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa66877-2922-4c47-bda6-dd57042ab3f0" containerName="revoke-keys"
Jul 21 04:56:47.146596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.146581    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9755d5cf-969d-4c98-b92d-4d2c0411c814" containerName="maas-api"
Jul 21 04:56:47.146596 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.146587    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="9755d5cf-969d-4c98-b92d-4d2c0411c814" containerName="maas-api"
Jul 21 04:56:47.146775 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.146664    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="9755d5cf-969d-4c98-b92d-4d2c0411c814" containerName="maas-api"
Jul 21 04:56:47.146775 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.146675    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa66877-2922-4c47-bda6-dd57042ab3f0" containerName="revoke-keys"
Jul 21 04:56:47.149840 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.149821    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.152243 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.152225    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6wgn4\"/\"default-dockercfg-kztqk\""
Jul 21 04:56:47.153297 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.153273    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6wgn4\"/\"openshift-service-ca.crt\""
Jul 21 04:56:47.153354 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.153311    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6wgn4\"/\"kube-root-ca.crt\""
Jul 21 04:56:47.157526 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.157036    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"]
Jul 21 04:56:47.227241 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.227212    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-proc\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.227439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.227247    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-sys\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.227439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.227294    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-podres\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.227439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.227368    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-665rr\" (UniqueName: \"kubernetes.io/projected/6c34852a-773c-44e0-8df3-069e2b7fbb6d-kube-api-access-665rr\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.227439 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.227421    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-lib-modules\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.327911 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.327873    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-proc\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.327911 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.327907    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-sys\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.328159 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.327944    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-podres\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.328159 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.327994    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-proc\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.328159 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.328009    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-sys\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.328159 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.328057    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-665rr\" (UniqueName: \"kubernetes.io/projected/6c34852a-773c-44e0-8df3-069e2b7fbb6d-kube-api-access-665rr\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.328159 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.328106    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-podres\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.328159 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.328132    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-lib-modules\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.328451 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.328260    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c34852a-773c-44e0-8df3-069e2b7fbb6d-lib-modules\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.338664 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.338633    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-665rr\" (UniqueName: \"kubernetes.io/projected/6c34852a-773c-44e0-8df3-069e2b7fbb6d-kube-api-access-665rr\") pod \"perf-node-gather-daemonset-w58m8\" (UID: \"6c34852a-773c-44e0-8df3-069e2b7fbb6d\") " pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.461418 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.461337    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.598051 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.598025    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"]
Jul 21 04:56:47.600295 ip-10-0-140-142 kubenswrapper[2546]: W0721 04:56:47.600258    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6c34852a_773c_44e0_8df3_069e2b7fbb6d.slice/crio-dd9de741f8806465ebf9d7bdd95964ea6f03268022d42e865b9167d46acb8bd5 WatchSource:0}: Error finding container dd9de741f8806465ebf9d7bdd95964ea6f03268022d42e865b9167d46acb8bd5: Status 404 returned error can't find the container with id dd9de741f8806465ebf9d7bdd95964ea6f03268022d42e865b9167d46acb8bd5
Jul 21 04:56:47.789366 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.789330    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8" event={"ID":"6c34852a-773c-44e0-8df3-069e2b7fbb6d","Type":"ContainerStarted","Data":"3a4b88682529c9c41ddabb3e43c5e5e7ca4152ff2a340a955213797c8eb12f48"}
Jul 21 04:56:47.789527 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.789380    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8" event={"ID":"6c34852a-773c-44e0-8df3-069e2b7fbb6d","Type":"ContainerStarted","Data":"dd9de741f8806465ebf9d7bdd95964ea6f03268022d42e865b9167d46acb8bd5"}
Jul 21 04:56:47.789527 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.789480    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:47.805581 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:47.805542    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8" podStartSLOduration=0.805528205 podStartE2EDuration="805.528205ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 04:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 04:56:47.80432704 +0000 UTC m=+2828.189815975" watchObservedRunningTime="2026-07-21 04:56:47.805528205 +0000 UTC m=+2828.191017125"
Jul 21 04:56:48.318788 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:48.318763    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-8qxcc_bda4a0b7-0c94-4219-acbc-50ccead235f2/download-server/0.log"
Jul 21 04:56:49.998432 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:49.998407    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qkv8k_2915ad8b-22a5-489c-ba33-3f592a81241a/dns/0.log"
Jul 21 04:56:50.019632 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:50.019602    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qkv8k_2915ad8b-22a5-489c-ba33-3f592a81241a/kube-rbac-proxy/0.log"
Jul 21 04:56:50.088133 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:50.088086    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-82nbl_2140b0ce-d757-4b6d-b07a-82e3b8e15f90/dns-node-resolver/0.log"
Jul 21 04:56:50.721639 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:50.721614    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vrmch_b761d554-5f1b-4da7-b197-d4cd24446e11/node-ca/0.log"
Jul 21 04:56:51.759106 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:51.759073    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hv9vk_791657e0-9d92-4cbc-8804-be0d29845b9a/discovery/0.log"
Jul 21 04:56:51.970710 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:51.970667    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-bgjkd_df0c9777-01ce-4a56-bb96-d5431297daa3/payload-pre-processing/0.log"
Jul 21 04:56:52.014287 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:52.014217    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-4fdww_33c1fcd2-e914-4377-854c-7d2459113f21/payload-processing/0.log"
Jul 21 04:56:52.718568 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:52.718543    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-99xh7_e2b54ec0-f78b-4aa6-a64f-041017187539/serve-healthcheck-canary/0.log"
Jul 21 04:56:53.300134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:53.300092    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnbk7_e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314/kube-rbac-proxy/0.log"
Jul 21 04:56:53.320498 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:53.320475    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnbk7_e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314/exporter/0.log"
Jul 21 04:56:53.342396 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:53.342372    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnbk7_e1d40ba1-7f18-42ee-a5d2-b4c2b1f69314/extractor/0.log"
Jul 21 04:56:53.804313 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:53.804288    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6wgn4/perf-node-gather-daemonset-w58m8"
Jul 21 04:56:55.910628 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:55.910591    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6ffdf96fc7-dknzk_b07b3a5d-dde8-462d-b3ed-02f20b685664/manager/0.log"
Jul 21 04:56:56.092502 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:56.092419    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d877cbcc8-5br7f_f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4/manager/0.log"
Jul 21 04:56:56.110964 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:56.110941    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d877cbcc8-5br7f_f739ef16-adf9-41d0-b0c0-e8c7ebb2e6b4/copy-manifests/0.log"
Jul 21 04:56:57.594141 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:57.594087    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hv9vk_791657e0-9d92-4cbc-8804-be0d29845b9a/discovery/0.log"
Jul 21 04:56:57.616451 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:57.616422    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-b844b6c94-c79g4_deaf9dfa-9998-4a2a-9898-fedf4ffbb784/manager/0.log"
Jul 21 04:56:57.666524 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:57.666498    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-6htbb_88c9f72f-8c5d-418c-93a7-44f4b53e4a99/openshift-lws-operator/0.log"
Jul 21 04:56:57.694229 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:57.694194    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-bgjkd_df0c9777-01ce-4a56-bb96-d5431297daa3/payload-pre-processing/0.log"
Jul 21 04:56:57.716528 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:57.716500    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-4fdww_33c1fcd2-e914-4377-854c-7d2459113f21/payload-processing/0.log"
Jul 21 04:56:59.103134 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:59.103088    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hv9vk_791657e0-9d92-4cbc-8804-be0d29845b9a/discovery/0.log"
Jul 21 04:56:59.214427 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:59.214353    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-bgjkd_df0c9777-01ce-4a56-bb96-d5431297daa3/payload-pre-processing/0.log"
Jul 21 04:56:59.243147 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:56:59.243098    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-4fdww_33c1fcd2-e914-4377-854c-7d2459113f21/payload-processing/0.log"
Jul 21 04:57:04.213703 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.213677    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5qqx_0037790b-0ec5-4bd1-ac34-35d830df61d3/kube-multus-additional-cni-plugins/0.log"
Jul 21 04:57:04.236857 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.236828    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5qqx_0037790b-0ec5-4bd1-ac34-35d830df61d3/egress-router-binary-copy/0.log"
Jul 21 04:57:04.257342 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.257318    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5qqx_0037790b-0ec5-4bd1-ac34-35d830df61d3/cni-plugins/0.log"
Jul 21 04:57:04.277679 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.277654    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5qqx_0037790b-0ec5-4bd1-ac34-35d830df61d3/bond-cni-plugin/0.log"
Jul 21 04:57:04.297997 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.297974    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5qqx_0037790b-0ec5-4bd1-ac34-35d830df61d3/routeoverride-cni/0.log"
Jul 21 04:57:04.320305 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.320280    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5qqx_0037790b-0ec5-4bd1-ac34-35d830df61d3/whereabouts-cni-bincopy/0.log"
Jul 21 04:57:04.341238 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.341210    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5qqx_0037790b-0ec5-4bd1-ac34-35d830df61d3/whereabouts-cni/0.log"
Jul 21 04:57:04.377698 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.377669    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gjjzt_3798e436-1a3e-47a4-8911-724c05dc7e53/kube-multus/0.log"
Jul 21 04:57:04.476617 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.476591    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-c6kf7_15bb69f1-6e84-48be-92fb-fc48362465b7/network-metrics-daemon/0.log"
Jul 21 04:57:04.496049 ip-10-0-140-142 kubenswrapper[2546]: I0721 04:57:04.496021    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-c6kf7_15bb69f1-6e84-48be-92fb-fc48362465b7/kube-rbac-proxy/0.log"