Jul 16 21:15:35.826381 ip-10-0-138-201 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 21:15:36.223782 ip-10-0-138-201 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:15:36.223782 ip-10-0-138-201 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 21:15:36.223782 ip-10-0-138-201 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:15:36.223782 ip-10-0-138-201 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 21:15:36.223782 ip-10-0-138-201 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:15:36.224477 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.223844    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 21:15:36.230887 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230864    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:36.230887 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230884    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:36.230887 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230888    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:36.230887 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230891    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:36.230887 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230895    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230898    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230901    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230904    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230907    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230911    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230915    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230918    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230921    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230923    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230927    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230930    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230933    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230937    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230941    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230944    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230947    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230949    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230953    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:36.231102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230956    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230958    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230961    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230964    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230966    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230969    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230971    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230976    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230978    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230981    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230984    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230986    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230989    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230992    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230994    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.230997    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231000    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231003    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231005    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231008    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:36.231572 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231010    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231013    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231015    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231018    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231020    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231023    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231025    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231028    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231031    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231035    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231037    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231040    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231042    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231045    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231047    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231050    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231052    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231055    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231057    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231060    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:36.232063 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231063    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231066    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231068    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231071    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231087    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231090    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231093    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231096    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231099    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231101    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231104    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231107    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231110    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231113    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231116    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231119    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231122    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231124    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231127    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231130    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:36.232601 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231133    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231136    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231139    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231142    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231152    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231155    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231161    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231164    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231166    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231169    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231172    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231174    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231177    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231179    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231183    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231187    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231923    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231930    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231933    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231937    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:36.233109 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231940    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231943    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231946    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231949    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231952    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231955    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231958    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231960    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231963    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231966    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231969    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231971    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231974    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231977    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231980    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231982    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231985    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231988    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231991    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231993    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:36.233629 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231996    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.231998    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232001    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232003    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232006    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232008    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232011    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232015    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232019    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232021    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232024    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232029    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232033    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232036    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232039    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232041    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232044    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232048    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232051    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:36.234185 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232054    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232057    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232060    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232063    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232066    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232069    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232072    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232097    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232101    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232105    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232110    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232114    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232117    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232120    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232122    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232125    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232127    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232130    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232133    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232135    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:36.234660 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232138    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232141    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232143    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232146    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232149    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232152    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232155    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232157    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232160    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232162    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232165    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232167    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232170    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232172    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232174    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232177    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232180    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232183    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232185    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232188    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:36.235174 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232191    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232194    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232196    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232199    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232201    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232203    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232206    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232208    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232211    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232213    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232216    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232219    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232221    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232223    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232226    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232228    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232602    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232607    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232610    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232613    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:36.235667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232616    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232619    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232621    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232624    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232626    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232629    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232632    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232634    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232637    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232640    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232642    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232645    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232648    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232651    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232653    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232656    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232658    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232661    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232663    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232665    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:36.236182 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232669    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232673    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232676    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232678    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232681    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232683    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232686    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232689    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232694    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232697    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232700    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232703    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232705    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232708    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232711    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232713    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232716    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232719    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232722    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:36.236671 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232726    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232729    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232732    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232734    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232737    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232739    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232742    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232745    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232748    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232750    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232759    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232762    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232764    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232767    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232770    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232773    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232775    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232778    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232781    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232783    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:36.237166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232786    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232788    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232791    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232794    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232796    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232798    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232801    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232804    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232806    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232809    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232811    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232814    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232818    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232820    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232823    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232825    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232828    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232830    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232837    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:36.237640 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232841    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232843    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232846    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232848    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232851    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232855    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232858    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232860    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232863    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232865    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232868    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232871    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232875    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232878    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232881    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232883    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.232886    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.232958    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.232965    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.232972    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 21:15:36.238102 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.232977    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.232981    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.232984    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.232989    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.232993    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.232998    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233001    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233004    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233007    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233010    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233013    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233016    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233019    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233023    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233026    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233031    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233034    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233038    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233041    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233044    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233048    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233051    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233054    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 21:15:36.238590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233057    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233061    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233063    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233066    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233070    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233088    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233093    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233096    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233098    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233101    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233104    2582 flags.go:64] FLAG: --enable-server="true"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233107    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233111    2582 flags.go:64] FLAG: --event-burst="100"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233114    2582 flags.go:64] FLAG: --event-qps="50"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233119    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233123    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233125    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233129    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233132    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233135    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233138    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233140    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233143    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233147    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233150    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 21:15:36.239174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233152    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233156    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233159    2582 flags.go:64] FLAG: --feature-gates=""
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233162    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233165    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233168    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233171    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233174    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233177    2582 flags.go:64] FLAG: --help="false"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233180    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233183    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233186    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233189    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233192    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233195    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233198    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233201    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233204    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233207    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233210    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233213    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233218    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233220    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233223    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 21:15:36.239927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233226    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233228    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233231    2582 flags.go:64] FLAG: --lock-file=""
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233234    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233237    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233240    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233245    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233249    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233252    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233255    2582 flags.go:64] FLAG: --logging-format="text"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233258    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233262    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233264    2582 flags.go:64] FLAG: --manifest-url=""
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233267    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233271    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233274    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233278    2582 flags.go:64] FLAG: --max-pods="110"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233281    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233285    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233287    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233290    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233293    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233296    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233299    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233306    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 21:15:36.240539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233309    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233312    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233315    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233318    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233322    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233327    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233330    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233333    2582 flags.go:64] FLAG: --port="10250"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233336    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233339    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0df564dd0ea216943"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233342    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233345    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233348    2582 flags.go:64] FLAG: --register-node="true"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233350    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233354    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233357    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233360    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233363    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233366    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233369    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233372    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233375    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233378    2582 flags.go:64] FLAG: --runonce="false"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233380    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233383    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 21:15:36.241166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233386    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233389    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233392    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233395    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233398    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233401    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233403    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233406    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233409    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233412    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233414    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233417    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233421    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233426    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233429    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233432    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233455    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233458    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233462    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233465    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233468    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233470    2582 flags.go:64] FLAG: --v="2"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233474    2582 flags.go:64] FLAG: --version="false"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233480    2582 flags.go:64] FLAG: --vmodule=""
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233484    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 21:15:36.241787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.233487    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233614    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233618    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233621    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233624    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233627    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233630    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233632    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233635    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233638    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233641    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233644    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233647    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233650    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233653    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233655    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233658    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233661    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233663    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233667    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:36.242410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233670    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233672    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233675    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233677    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233679    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233682    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233685    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233687    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233690    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233693    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233696    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233699    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233702    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233704    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233707    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233709    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233712    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233714    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233716    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233719    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:36.242895 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233721    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233724    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233726    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233729    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233731    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233733    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233736    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233738    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233741    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233743    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233746    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233750    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233753    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233755    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233758    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233760    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233763    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233766    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233768    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233771    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:36.243452 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233773    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233777    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233780    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233782    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233785    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233788    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233790    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233793    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233796    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233798    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233800    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233803    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233805    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233808    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233812    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233816    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233818    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233821    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233823    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:36.243937 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233826    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233828    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233831    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233834    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233839    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233842    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233845    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233848    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233851    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233854    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233856    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233859    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233861    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233864    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233868    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233872    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233875    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233878    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233881    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:36.244426 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233884    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.233887    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.234642    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.241768    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.241785    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.241805    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.241811    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241866    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241872    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241877    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241880    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241883    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241886    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241889    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:36.244889 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241893    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241895    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241898    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241901    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241903    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241906    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241908    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241911    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241914    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241916    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241919    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241921    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241924    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241927    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241930    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241932    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241935    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241938    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241941    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241943    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:36.245259 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241946    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241949    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241952    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241955    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241957    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241960    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241963    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241965    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241968    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241970    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241973    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241975    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241978    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241980    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241983    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241985    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241988    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241992    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241995    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.241998    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:36.245755 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242001    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242004    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242006    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242009    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242012    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242015    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242018    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242021    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242023    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242026    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242029    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242032    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242035    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242037    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242041    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242043    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242047    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242050    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242052    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242055    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:36.246254 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242058    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242060    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242063    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242065    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242068    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242070    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242089    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242092    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242095    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242098    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242100    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242103    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242106    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242108    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242111    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242114    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242116    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242119    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242122    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242125    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:36.246773 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242127    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242130    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242133    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242136    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242139    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242141    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242144    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242147    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242150    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242152    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242154    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242158    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.242164    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242282    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242286    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:36.247284 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242289    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242292    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242294    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242297    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242299    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242302    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242304    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242307    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242309    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242312    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242315    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242317    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242319    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242322    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242324    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242327    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242330    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242332    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242335    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242338    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:36.247667 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242341    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242344    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242347    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242349    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242352    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242354    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242357    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242359    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242363    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242366    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242368    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242371    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242373    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242376    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242378    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242381    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242383    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242385    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242388    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:36.248166 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242391    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242393    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242395    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242398    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242400    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242403    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242405    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242408    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242410    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242413    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242416    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242419    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242422    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242425    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242429    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242433    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242436    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242438    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242441    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:36.248635 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242443    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242446    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242449    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242451    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242454    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242456    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242459    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242461    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242464    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242466    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242469    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242471    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242474    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242477    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242479    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242482    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242484    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242487    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242490    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242492    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:36.249168 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242494    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242499    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242502    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242505    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242508    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242510    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242513    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242516    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242518    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242521    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242523    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242526    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242528    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242531    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242533    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242535    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242538    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242540    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:36.249672 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:36.242543    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:36.250123 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.242548    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:15:36.250123 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.242648    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 21:15:36.250123 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.244987    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 21:15:36.250123 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.246656    2582 server.go:1019] "Starting client certificate rotation"
Jul 16 21:15:36.250123 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.246759    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 21:15:36.250123 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.246790    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 21:15:36.250123 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.249284    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 21:15:36.252262 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.252246    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 21:15:36.263633 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.263615    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 16 21:15:36.266161 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.266142    2582 log.go:25] "Validated CRI v1 image API"
Jul 16 21:15:36.266904 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.266889    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 21:15:36.269164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.269122    2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9626004e-b28f-4348-b6ed-3ff1f10de6e2:/dev/nvme0n1p3 9b90a69e-9488-40d2-b57e-b9345f622f51:/dev/nvme0n1p4]
Jul 16 21:15:36.269164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.269159    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 21:15:36.274978 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.274874    2582 manager.go:216] Machine: {Timestamp:2026-07-16 21:15:36.273472802 +0000 UTC m=+0.330303708 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3067016 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2713d047a2f25e0d26acec45de603a SystemUUID:ec2713d0-47a2-f25e-0d26-acec45de603a BootID:d87202e1-dc09-4581-924a-7a486ca70c60 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8c:a8:22:2b:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8c:a8:22:2b:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:d9:bc:c5:52:69 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 21:15:36.274978 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.274975    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 21:15:36.275094 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.275053    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 21:15:36.278114 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.278091    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 21:15:36.278249 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.278116    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-201.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 21:15:36.278755 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.278745    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 21:15:36.278786 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.278758    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 21:15:36.278786 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.278766    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 21:15:36.279521 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.279510    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 21:15:36.281292 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.281282    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 21:15:36.285169 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.285158    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 21:15:36.285276 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.285266    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 21:15:36.285873 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.285864    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 21:15:36.285907 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.285877    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 21:15:36.285907 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.285887    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 21:15:36.285907 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.285896    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 16 21:15:36.285907 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.285904    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 21:15:36.286241 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.286230    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 21:15:36.286278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.286249    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 21:15:36.288414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.288392    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 21:15:36.290290 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.290277    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 21:15:36.290328 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.290307    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 21:15:36.291649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291636    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 21:15:36.291693 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291656    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 21:15:36.291693 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291662    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 21:15:36.291693 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291668    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 21:15:36.291693 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291677    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 21:15:36.291693 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291683    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 21:15:36.291693 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291690    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 21:15:36.291855 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291696    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 21:15:36.291855 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291711    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 21:15:36.291855 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291717    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 21:15:36.291855 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291729    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 21:15:36.291855 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.291735    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 21:15:36.292527 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.292518    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 21:15:36.292527 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.292527    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 21:15:36.295753 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.295738    2582 server.go:1268] "Started kubelet"
Jul 16 21:15:36.295868 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.295821    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 21:15:36.295914 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.295893    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 21:15:36.296053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.296018    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 21:15:36.296650 ip-10-0-138-201 systemd[1]: Started Kubernetes Kubelet.
Jul 16 21:15:36.300909 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.300882    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 21:15:36.301446 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.301412    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-201.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 21:15:36.301682 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.301655    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 21:15:36.301777 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.301755    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-201.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 21:15:36.303632 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.303615    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 21:15:36.303734 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.303706    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 21:15:36.309928 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.309914    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 21:15:36.309979 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.309934    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 21:15:36.310368 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.310033    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:36.310737 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.310720    2582 factory.go:55] Registering systemd factory
Jul 16 21:15:36.310824 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.310741    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 16 21:15:36.310824 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.310750    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 21:15:36.310824 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.310756    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 21:15:36.310824 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.310775    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 21:15:36.310971 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.310960    2582 factory.go:153] Registering CRI-O factory
Jul 16 21:15:36.311017 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.310976    2582 factory.go:223] Registration of the crio container factory successfully
Jul 16 21:15:36.311017 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.311011    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 21:15:36.311099 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.311020    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 21:15:36.311099 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.311039    2582 factory.go:103] Registering Raw factory
Jul 16 21:15:36.311099 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.311049    2582 manager.go:1196] Started watching for new ooms in manager
Jul 16 21:15:36.311309 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.311021    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 21:15:36.311462 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.311448    2582 manager.go:349] Starting recovery of all containers
Jul 16 21:15:36.311513 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.310468    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-201.ec2.internal.18c2e1ad2d2dd8e5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-201.ec2.internal,UID:ip-10-0-138-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-201.ec2.internal,},FirstTimestamp:2026-07-16 21:15:36.295708901 +0000 UTC m=+0.352539804,LastTimestamp:2026-07-16 21:15:36.295708901 +0000 UTC m=+0.352539804,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-201.ec2.internal,}"
Jul 16 21:15:36.320927 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.320892    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 21:15:36.320927 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.320872    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-201.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 21:15:36.321794 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.321764    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 21:15:36.323115 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.323093    2582 manager.go:354] Recovery completed
Jul 16 21:15:36.327818 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.327804    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:36.329649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.329629    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:36.329735 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.329664    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:36.329735 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.329676    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:36.330617 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.330604    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 21:15:36.330617 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.330617    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 21:15:36.330705 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.330634    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 21:15:36.330705 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.330643    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:36.331417 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.331404    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:36.331473 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.331427    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:36.331473 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.331438    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:36.332262 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.332251    2582 policy_none.go:49] "None policy: Start"
Jul 16 21:15:36.332308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.332266    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 21:15:36.332308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.332276    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 21:15:36.333222 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.333148    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-201.ec2.internal.18c2e1ad2f33b9fd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-201.ec2.internal,UID:ip-10-0-138-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-201.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-201.ec2.internal,},FirstTimestamp:2026-07-16 21:15:36.329648637 +0000 UTC m=+0.386479548,LastTimestamp:2026-07-16 21:15:36.329648637 +0000 UTC m=+0.386479548,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-201.ec2.internal,}"
Jul 16 21:15:36.333442 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.333429    2582 policy_none.go:47] "Start"
Jul 16 21:15:36.347181 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.346983    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-201.ec2.internal.18c2e1ad2f340c11  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-201.ec2.internal,UID:ip-10-0-138-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-201.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-201.ec2.internal,},FirstTimestamp:2026-07-16 21:15:36.329669649 +0000 UTC m=+0.386500552,LastTimestamp:2026-07-16 21:15:36.329669649 +0000 UTC m=+0.386500552,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-201.ec2.internal,}"
Jul 16 21:15:36.355404 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.354815    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-201.ec2.internal.18c2e1ad2f3439e6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-201.ec2.internal,UID:ip-10-0-138-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-201.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-201.ec2.internal,},FirstTimestamp:2026-07-16 21:15:36.329681382 +0000 UTC m=+0.386512287,LastTimestamp:2026-07-16 21:15:36.329681382 +0000 UTC m=+0.386512287,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-201.ec2.internal,}"
Jul 16 21:15:36.363565 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.363549    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tcpgq"
Jul 16 21:15:36.364309 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.364243    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-201.ec2.internal.18c2e1ad2f33b9fd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-201.ec2.internal.18c2e1ad2f33b9fd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-201.ec2.internal,UID:ip-10-0-138-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-201.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-201.ec2.internal,},FirstTimestamp:2026-07-16 21:15:36.329648637 +0000 UTC m=+0.386479548,LastTimestamp:2026-07-16 21:15:36.331415963 +0000 UTC m=+0.388246867,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-201.ec2.internal,}"
Jul 16 21:15:36.373660 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.373646    2582 manager.go:337] "Starting Device Plugin manager"
Jul 16 21:15:36.373747 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.373677    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 21:15:36.373747 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.373687    2582 server.go:87] "Starting device plugin registration server"
Jul 16 21:15:36.373904 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.373893    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 21:15:36.373955 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.373905    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 21:15:36.374019 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.374003    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 21:15:36.374114 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.374097    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 21:15:36.374114 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.374112    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 21:15:36.374621 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.374603    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 21:15:36.374701 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.374649    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:36.376128 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.376045    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-201.ec2.internal.18c2e1ad2f340c11\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-201.ec2.internal.18c2e1ad2f340c11  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-201.ec2.internal,UID:ip-10-0-138-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-201.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-201.ec2.internal,},FirstTimestamp:2026-07-16 21:15:36.329669649 +0000 UTC m=+0.386500552,LastTimestamp:2026-07-16 21:15:36.331431957 +0000 UTC m=+0.388262859,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-201.ec2.internal,}"
Jul 16 21:15:36.379953 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.379873    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-201.ec2.internal.18c2e1ad2f3439e6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-201.ec2.internal.18c2e1ad2f3439e6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-201.ec2.internal,UID:ip-10-0-138-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-201.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-201.ec2.internal,},FirstTimestamp:2026-07-16 21:15:36.329681382 +0000 UTC m=+0.386512287,LastTimestamp:2026-07-16 21:15:36.331442023 +0000 UTC m=+0.388272925,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-201.ec2.internal,}"
Jul 16 21:15:36.383199 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.383184    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tcpgq"
Jul 16 21:15:36.396010 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.395949    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-201.ec2.internal.18c2e1ad31f37d82  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-201.ec2.internal,UID:ip-10-0-138-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-138-201.ec2.internal,},FirstTimestamp:2026-07-16 21:15:36.375770498 +0000 UTC m=+0.432601388,LastTimestamp:2026-07-16 21:15:36.375770498 +0000 UTC m=+0.432601388,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-201.ec2.internal,}"
Jul 16 21:15:36.474328 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.474276    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:36.475750 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.475210    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:36.475750 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.475241    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:36.475750 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.475251    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:36.475750 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.475277    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.498886 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.498851    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 21:15:36.500221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.500203    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 21:15:36.500312 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.500241    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 21:15:36.500312 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.500261    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 21:15:36.500312 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.500294    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 21:15:36.515616 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.515598    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.515701 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.515621    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-201.ec2.internal\": node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:36.515868 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.515854    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:36.600521 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.600494    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal"]
Jul 16 21:15:36.600638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.600573    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:36.601446 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.601429    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:36.601538 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.601459    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:36.601538 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.601470    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:36.603033 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.603021    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:36.603168 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.603154    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.603208 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.603187    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:36.603742 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.603725    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:36.603841 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.603756    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:36.603841 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.603770    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:36.603841 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.603733    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:36.603841 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.603817    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:36.603841 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.603827    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:36.605240 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.605225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.605300 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.605253    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:36.605942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.605924    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:36.606022 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.605956    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:36.606022 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.605969    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:36.612191 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.612172    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4486b65632fc154b9212b0f0a3586dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal\" (UID: \"f4486b65632fc154b9212b0f0a3586dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.612254 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.612198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33cd532730a76493a09bd77465fc98c4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-201.ec2.internal\" (UID: \"33cd532730a76493a09bd77465fc98c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.612254 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.612213    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f4486b65632fc154b9212b0f0a3586dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal\" (UID: \"f4486b65632fc154b9212b0f0a3586dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.630631 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.630612    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-201.ec2.internal\" not found" node="ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.634364 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.634349    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-201.ec2.internal\" not found" node="ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.635572 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.635554    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:36.713128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.713101    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33cd532730a76493a09bd77465fc98c4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-201.ec2.internal\" (UID: \"33cd532730a76493a09bd77465fc98c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.713128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.713129    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f4486b65632fc154b9212b0f0a3586dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal\" (UID: \"f4486b65632fc154b9212b0f0a3586dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.713266 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.713147    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4486b65632fc154b9212b0f0a3586dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal\" (UID: \"f4486b65632fc154b9212b0f0a3586dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.713266 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.713184    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4486b65632fc154b9212b0f0a3586dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal\" (UID: \"f4486b65632fc154b9212b0f0a3586dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.713266 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.713203    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33cd532730a76493a09bd77465fc98c4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-201.ec2.internal\" (UID: \"33cd532730a76493a09bd77465fc98c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.713266 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.713213    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f4486b65632fc154b9212b0f0a3586dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal\" (UID: \"f4486b65632fc154b9212b0f0a3586dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.736161 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.736117    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:36.836881 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.836847    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:36.933371 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.933340    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:36.938687 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:36.937761    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:36.939938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:36.939918    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:37.040133 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:37.040031    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:37.141185 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:37.141140    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:37.242447 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:37.242404    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:37.246917 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.246896    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 21:15:37.247069 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.247046    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 21:15:37.342613 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:37.342538    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:37.389287 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.389239    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 21:10:37 +0000 UTC" deadline="2028-04-07 21:44:03.539720472 +0000 UTC"
Jul 16 21:15:37.389287 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.389280    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15144h28m26.150444695s"
Jul 16 21:15:37.441047 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.441019    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:37.442600 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:37.442580    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:37.443167 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.443136    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:37.443167 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.443167    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:37.443307 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.443178    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:37.543535 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:37.543484    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-201.ec2.internal\" not found"
Jul 16 21:15:37.602745 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:37.602665    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4486b65632fc154b9212b0f0a3586dc.slice/crio-d9b14fc2c37208b4dcf01e0b6bd0fe3f1a4cd3bfc91455344c427c7f498a30d8 WatchSource:0}: Error finding container d9b14fc2c37208b4dcf01e0b6bd0fe3f1a4cd3bfc91455344c427c7f498a30d8: Status 404 returned error can't find the container with id d9b14fc2c37208b4dcf01e0b6bd0fe3f1a4cd3bfc91455344c427c7f498a30d8
Jul 16 21:15:37.608244 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.608226    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:15:37.617240 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.617219    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:37.711749 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.711708    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:37.712135 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:37.712111    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33cd532730a76493a09bd77465fc98c4.slice/crio-2e54ebbb98f0f36ed1ef31052799b4e53e7d0f53e5def5c6a00197fb637b52e3 WatchSource:0}: Error finding container 2e54ebbb98f0f36ed1ef31052799b4e53e7d0f53e5def5c6a00197fb637b52e3: Status 404 returned error can't find the container with id 2e54ebbb98f0f36ed1ef31052799b4e53e7d0f53e5def5c6a00197fb637b52e3
Jul 16 21:15:37.746237 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.746209    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 21:15:37.748102 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.748070    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal"
Jul 16 21:15:37.767349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.767329    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 21:15:37.837230 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.837195    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:37.841586 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:37.841556    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:38.287916 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.287872    2582 apiserver.go:52] "Watching apiserver"
Jul 16 21:15:38.298949 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.298914    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 21:15:38.301430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.301393    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-hlv9v","kube-system/konnectivity-agent-657qw","openshift-image-registry/node-ca-64prw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal","openshift-network-diagnostics/network-check-target-kqszq","openshift-network-operator/iptables-alerter-2886f","openshift-ovn-kubernetes/ovnkube-node-84xhp","kube-system/global-pull-secret-syncer-h98vh","kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w","openshift-cluster-node-tuning-operator/tuned-xl8zf","openshift-dns/node-resolver-772xx","openshift-multus/multus-5bj9d","openshift-multus/multus-additional-cni-plugins-xzvc6"]
Jul 16 21:15:38.303768 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.303739    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.307076 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.307053    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.308800 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.308776    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 21:15:38.309774 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.309389    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.311413 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.310859    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 21:15:38.311529 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.311504    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.314232 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.314207    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:38.314344 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.314315    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:38.315451 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.315431    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.317521 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.317496    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 21:15:38.317876 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.317852    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:38.317975 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.317932    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:38.319723 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.319700    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.321032 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.321011    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:15:38.321217 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.321189    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-glk7c\""
Jul 16 21:15:38.321781 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.321468    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 21:15:38.321781 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.321548    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:15:38.322553 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.322531    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 21:15:38.323516 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323493    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-run-multus-certs\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.323634 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-os-release\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.323634 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323548    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:38.323634 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323565    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c63fecfd-97f0-49e3-b741-c240ed7357ac-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.323634 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323590    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-socket-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.323634 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-conf-dir\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.323845 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.323664    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:38.323845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-daemon-config\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.323845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323705    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-device-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.323845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6twl\" (UniqueName: \"kubernetes.io/projected/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-kube-api-access-k6twl\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.323845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323747    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-system-cni-dir\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.323845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-system-cni-dir\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.323845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323786    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-registration-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.323845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323812    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-etc-selinux\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-var-lib-cni-multus\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323892    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-etc-kubernetes\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323921    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h9jn\" (UniqueName: \"kubernetes.io/projected/c66b7d07-c6ac-49cf-9841-425d060e4222-kube-api-access-7h9jn\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323949    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-sys-fs\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.323995    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-cni-dir\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324019    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c66b7d07-c6ac-49cf-9841-425d060e4222-cni-binary-copy\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324044    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-socket-dir-parent\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324074    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c63fecfd-97f0-49e3-b741-c240ed7357ac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324103    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-run-k8s-cni-cncf-io\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324141    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-run-netns\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/063fca31-aa29-467d-b4f2-0a81b45d9707-host\") pod \"node-ca-64prw\" (UID: \"063fca31-aa29-467d-b4f2-0a81b45d9707\") " pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324257    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c63fecfd-97f0-49e3-b741-c240ed7357ac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.324281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324284    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/063fca31-aa29-467d-b4f2-0a81b45d9707-serviceca\") pod \"node-ca-64prw\" (UID: \"063fca31-aa29-467d-b4f2-0a81b45d9707\") " pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92h9t\" (UniqueName: \"kubernetes.io/projected/063fca31-aa29-467d-b4f2-0a81b45d9707-kube-api-access-92h9t\") pod \"node-ca-64prw\" (UID: \"063fca31-aa29-467d-b4f2-0a81b45d9707\") " pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324356    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-cnibin\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324415    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpqw8\" (UniqueName: \"kubernetes.io/projected/c63fecfd-97f0-49e3-b741-c240ed7357ac-kube-api-access-fpqw8\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324444    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-cnibin\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324490    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-var-lib-kubelet\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324514    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-hostroot\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324523    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324587    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324635    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-os-release\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324657    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324661    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-var-lib-cni-bin\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.324910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.324749    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 21:15:38.326529 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.326507    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 21:15:38.326644 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.326542    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 21:15:38.326853 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.326789    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 21:15:38.326934 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.326905    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q4qlx\""
Jul 16 21:15:38.327109 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.327084    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-khpmk\""
Jul 16 21:15:38.327303 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.327285    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rk2l4\""
Jul 16 21:15:38.327375 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.327305    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 21:15:38.327998 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.327962    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cfdgr\""
Jul 16 21:15:38.328194 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.328176    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 21:15:38.328328 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.328309    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wzqz2\""
Jul 16 21:15:38.328536 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.327970    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 21:15:38.328638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.328623    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 21:15:38.328869 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.328852    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gqv4l\""
Jul 16 21:15:38.328919 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.328875    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 21:15:38.329077 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.329054    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 21:15:38.329217 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.329113    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 21:15:38.329288 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.329219    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.329357 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.329339    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 21:15:38.329491 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.329471    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.329548 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.329523    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 21:15:38.329626 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.329611    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 21:15:38.344827 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.344803    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 21:15:38.353705 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.353070    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:15:38.359318 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.359297    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 21:15:38.360086 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.359979    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bngwc\""
Jul 16 21:15:38.360351 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.360331    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 21:15:38.367175 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.367144    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mjjgg\""
Jul 16 21:15:38.368877 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.368856    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 21:15:38.369027 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.369003    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 21:15:38.370752 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.370559    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 21:15:38.384868 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.384846    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d6msb"
Jul 16 21:15:38.401702 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.401669    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d6msb"
Jul 16 21:15:38.412565 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.412542    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 21:15:38.425276 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425245    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/063fca31-aa29-467d-b4f2-0a81b45d9707-serviceca\") pod \"node-ca-64prw\" (UID: \"063fca31-aa29-467d-b4f2-0a81b45d9707\") " pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.425407 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92h9t\" (UniqueName: \"kubernetes.io/projected/063fca31-aa29-467d-b4f2-0a81b45d9707-kube-api-access-92h9t\") pod \"node-ca-64prw\" (UID: \"063fca31-aa29-467d-b4f2-0a81b45d9707\") " pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.425407 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425318    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-hostroot\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.425407 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-tuned\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.425407 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc9r8\" (UniqueName: \"kubernetes.io/projected/892e666e-1ad3-4df7-a721-7ecf591e8868-kube-api-access-cc9r8\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.425407 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425384    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cmxl\" (UniqueName: \"kubernetes.io/projected/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-kube-api-access-4cmxl\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425410    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-var-lib-cni-bin\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425428    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2781ca06-e247-4b8b-879e-72a32dbdc0bf-agent-certs\") pod \"konnectivity-agent-657qw\" (UID: \"2781ca06-e247-4b8b-879e-72a32dbdc0bf\") " pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425454    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-os-release\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425470    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c63fecfd-97f0-49e3-b741-c240ed7357ac-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-conf-dir\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425501    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-run-systemd\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425517    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-var-lib-openvswitch\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-cni-bin\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf351621-f759-4dd6-8e96-0b52739329da-ovnkube-script-lib\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425581    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-device-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6twl\" (UniqueName: \"kubernetes.io/projected/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-kube-api-access-k6twl\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425625    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-system-cni-dir\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-etc-openvswitch\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425660    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-run-openvswitch\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.425670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425675    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-lib-modules\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425696    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-system-cni-dir\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-registration-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425736    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-etc-selinux\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425751    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-var-lib-cni-multus\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-etc-kubernetes\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425795    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h9jn\" (UniqueName: \"kubernetes.io/projected/c66b7d07-c6ac-49cf-9841-425d060e4222-kube-api-access-7h9jn\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.425794    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/063fca31-aa29-467d-b4f2-0a81b45d9707-serviceca\") pod \"node-ca-64prw\" (UID: \"063fca31-aa29-467d-b4f2-0a81b45d9707\") " pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426051    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb-host-slash\") pod \"iptables-alerter-2886f\" (UID: \"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb\") " pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426099    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-kubernetes\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-cni-dir\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426177    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-socket-dir-parent\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426182    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-hostroot\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426256    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-cni-dir\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426313    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-device-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-system-cni-dir\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426344    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-var-lib-cni-bin\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426377    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-var-lib-cni-multus\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426379    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-system-cni-dir\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426411    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-etc-kubernetes\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-sys\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426443    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-etc-selinux\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426478    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426495    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e047fbb7-6b0e-4ad1-8348-de32e09d26ad-hosts-file\") pod \"node-resolver-772xx\" (UID: \"e047fbb7-6b0e-4ad1-8348-de32e09d26ad\") " pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426524    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-run-netns\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426537    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-os-release\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426542    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-run-netns\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426563    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-modprobe-d\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426582    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-sysctl-conf\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426605    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/def9cc97-3b3b-455a-a095-21a14521bb12-dbus\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf351621-f759-4dd6-8e96-0b52739329da-ovn-node-metrics-cert\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426671    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2lnl\" (UniqueName: \"kubernetes.io/projected/e047fbb7-6b0e-4ad1-8348-de32e09d26ad-kube-api-access-j2lnl\") pod \"node-resolver-772xx\" (UID: \"e047fbb7-6b0e-4ad1-8348-de32e09d26ad\") " pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426695    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-cnibin\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.426945 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426715    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpqw8\" (UniqueName: \"kubernetes.io/projected/c63fecfd-97f0-49e3-b741-c240ed7357ac-kube-api-access-fpqw8\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426735    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-cnibin\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426752    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-var-lib-kubelet\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb-iptables-alerter-script\") pod \"iptables-alerter-2886f\" (UID: \"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb\") " pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426793    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-run-ovn-kubernetes\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-cni-netd\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426838    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426861    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-os-release\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426900    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-run-multus-certs\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-node-log\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426923    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c63fecfd-97f0-49e3-b741-c240ed7357ac-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426939    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf351621-f759-4dd6-8e96-0b52739329da-env-overrides\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-conf-dir\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.426960    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-host\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427042    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-socket-dir-parent\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427047    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-run-netns\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.427840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427097    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-socket-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427101    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-registration-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427122    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-daemon-config\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427145    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-kubelet\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427186    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7kvt\" (UniqueName: \"kubernetes.io/projected/a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb-kube-api-access-g7kvt\") pod \"iptables-alerter-2886f\" (UID: \"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb\") " pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427213    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-slash\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427214    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-socket-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427249    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2781ca06-e247-4b8b-879e-72a32dbdc0bf-konnectivity-ca\") pod \"konnectivity-agent-657qw\" (UID: \"2781ca06-e247-4b8b-879e-72a32dbdc0bf\") " pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427299    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-cnibin\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c63fecfd-97f0-49e3-b741-c240ed7357ac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427453    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427492    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-os-release\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427695    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-run-multus-certs\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427740    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-cnibin\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427766    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-var-lib-kubelet\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427789    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-sysconfig\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427816    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-sysctl-d\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.428631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427842    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-var-lib-kubelet\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427863    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/892e666e-1ad3-4df7-a721-7ecf591e8868-tmp\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427900    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427922    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c66b7d07-c6ac-49cf-9841-425d060e4222-multus-daemon-config\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427924    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-systemd-units\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.427978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-run-ovn\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428011    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/def9cc97-3b3b-455a-a095-21a14521bb12-kubelet-config\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e047fbb7-6b0e-4ad1-8348-de32e09d26ad-tmp-dir\") pod \"node-resolver-772xx\" (UID: \"e047fbb7-6b0e-4ad1-8348-de32e09d26ad\") " pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-sys-fs\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428152    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-sys-fs\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428193    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c66b7d07-c6ac-49cf-9841-425d060e4222-cni-binary-copy\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428236    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-log-socket\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428272    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf351621-f759-4dd6-8e96-0b52739329da-ovnkube-config\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428388    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-systemd\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428449    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c63fecfd-97f0-49e3-b741-c240ed7357ac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428503    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-run-k8s-cni-cncf-io\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95wpl\" (UniqueName: \"kubernetes.io/projected/cf351621-f759-4dd6-8e96-0b52739329da-kube-api-access-95wpl\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.429410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428581    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c66b7d07-c6ac-49cf-9841-425d060e4222-host-run-k8s-cni-cncf-io\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.430349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428634    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/063fca31-aa29-467d-b4f2-0a81b45d9707-host\") pod \"node-ca-64prw\" (UID: \"063fca31-aa29-467d-b4f2-0a81b45d9707\") " pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.430349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c63fecfd-97f0-49e3-b741-c240ed7357ac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.430349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428703    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.430349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428709    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c66b7d07-c6ac-49cf-9841-425d060e4222-cni-binary-copy\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.430349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428730    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/063fca31-aa29-467d-b4f2-0a81b45d9707-host\") pod \"node-ca-64prw\" (UID: \"063fca31-aa29-467d-b4f2-0a81b45d9707\") " pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.430349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.428731    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-run\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.430349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.429036    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c63fecfd-97f0-49e3-b741-c240ed7357ac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.430349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.429169    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c63fecfd-97f0-49e3-b741-c240ed7357ac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.461932 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.461904    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 21:15:38.466466 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.466440    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92h9t\" (UniqueName: \"kubernetes.io/projected/063fca31-aa29-467d-b4f2-0a81b45d9707-kube-api-access-92h9t\") pod \"node-ca-64prw\" (UID: \"063fca31-aa29-467d-b4f2-0a81b45d9707\") " pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.466606 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.466469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6twl\" (UniqueName: \"kubernetes.io/projected/98ef19e0-985a-4e7e-b9b9-e7a3b33b9084-kube-api-access-k6twl\") pod \"aws-ebs-csi-driver-node-bdh2w\" (UID: \"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.466606 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.466438    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpqw8\" (UniqueName: \"kubernetes.io/projected/c63fecfd-97f0-49e3-b741-c240ed7357ac-kube-api-access-fpqw8\") pod \"multus-additional-cni-plugins-xzvc6\" (UID: \"c63fecfd-97f0-49e3-b741-c240ed7357ac\") " pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.471958 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.471927    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h9jn\" (UniqueName: \"kubernetes.io/projected/c66b7d07-c6ac-49cf-9841-425d060e4222-kube-api-access-7h9jn\") pod \"multus-5bj9d\" (UID: \"c66b7d07-c6ac-49cf-9841-425d060e4222\") " pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.506396 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.506339    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal" event={"ID":"33cd532730a76493a09bd77465fc98c4","Type":"ContainerStarted","Data":"2e54ebbb98f0f36ed1ef31052799b4e53e7d0f53e5def5c6a00197fb637b52e3"}
Jul 16 21:15:38.508148 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.508113    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal" event={"ID":"f4486b65632fc154b9212b0f0a3586dc","Type":"ContainerStarted","Data":"d9b14fc2c37208b4dcf01e0b6bd0fe3f1a4cd3bfc91455344c427c7f498a30d8"}
Jul 16 21:15:38.529097 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529033    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:38.529097 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529075    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:38.529342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529118    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e047fbb7-6b0e-4ad1-8348-de32e09d26ad-hosts-file\") pod \"node-resolver-772xx\" (UID: \"e047fbb7-6b0e-4ad1-8348-de32e09d26ad\") " pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.529342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-run-netns\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.529342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-modprobe-d\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.529342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-sysctl-conf\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.529342 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.529245    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:38.529342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529260    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/def9cc97-3b3b-455a-a095-21a14521bb12-dbus\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:38.529342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529288    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf351621-f759-4dd6-8e96-0b52739329da-ovn-node-metrics-cert\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.529342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529298    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-run-netns\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.529342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529313    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2lnl\" (UniqueName: \"kubernetes.io/projected/e047fbb7-6b0e-4ad1-8348-de32e09d26ad-kube-api-access-j2lnl\") pod \"node-resolver-772xx\" (UID: \"e047fbb7-6b0e-4ad1-8348-de32e09d26ad\") " pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529357    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb-iptables-alerter-script\") pod \"iptables-alerter-2886f\" (UID: \"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb\") " pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529379    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-modprobe-d\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.529401    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs podName:0265e540-9cd6-45da-aa44-bc5d8a5a9a42 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:39.029365154 +0000 UTC m=+3.086196063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs") pod "network-metrics-daemon-hlv9v" (UID: "0265e540-9cd6-45da-aa44-bc5d8a5a9a42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.529247    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529436    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/def9cc97-3b3b-455a-a095-21a14521bb12-dbus\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.529461    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret podName:def9cc97-3b3b-455a-a095-21a14521bb12 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:39.029444641 +0000 UTC m=+3.086275535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret") pod "global-pull-secret-syncer-h98vh" (UID: "def9cc97-3b3b-455a-a095-21a14521bb12") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529435    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-run-ovn-kubernetes\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529507    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-cni-netd\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-node-log\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529575    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf351621-f759-4dd6-8e96-0b52739329da-env-overrides\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-host\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529595    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-sysctl-conf\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529261    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e047fbb7-6b0e-4ad1-8348-de32e09d26ad-hosts-file\") pod \"node-resolver-772xx\" (UID: \"e047fbb7-6b0e-4ad1-8348-de32e09d26ad\") " pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.529649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529651    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-node-log\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529655    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-run-ovn-kubernetes\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529687    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-cni-netd\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529617    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-kubelet\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529715    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7kvt\" (UniqueName: \"kubernetes.io/projected/a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb-kube-api-access-g7kvt\") pod \"iptables-alerter-2886f\" (UID: \"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb\") " pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529734    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-slash\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2781ca06-e247-4b8b-879e-72a32dbdc0bf-konnectivity-ca\") pod \"konnectivity-agent-657qw\" (UID: \"2781ca06-e247-4b8b-879e-72a32dbdc0bf\") " pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-sysconfig\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529785    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-sysctl-d\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529801    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-var-lib-kubelet\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/892e666e-1ad3-4df7-a721-7ecf591e8868-tmp\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529833    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529848    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-systemd-units\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-run-ovn\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/def9cc97-3b3b-455a-a095-21a14521bb12-kubelet-config\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529914    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e047fbb7-6b0e-4ad1-8348-de32e09d26ad-tmp-dir\") pod \"node-resolver-772xx\" (UID: \"e047fbb7-6b0e-4ad1-8348-de32e09d26ad\") " pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529934    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-log-socket\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.530238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529950    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf351621-f759-4dd6-8e96-0b52739329da-ovnkube-config\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529965    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-systemd\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.529985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95wpl\" (UniqueName: \"kubernetes.io/projected/cf351621-f759-4dd6-8e96-0b52739329da-kube-api-access-95wpl\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530034    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-run\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530038    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb-iptables-alerter-script\") pod \"iptables-alerter-2886f\" (UID: \"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb\") " pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530054    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-tuned\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc9r8\" (UniqueName: \"kubernetes.io/projected/892e666e-1ad3-4df7-a721-7ecf591e8868-kube-api-access-cc9r8\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530103    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf351621-f759-4dd6-8e96-0b52739329da-env-overrides\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530196    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cmxl\" (UniqueName: \"kubernetes.io/projected/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-kube-api-access-4cmxl\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2781ca06-e247-4b8b-879e-72a32dbdc0bf-agent-certs\") pod \"konnectivity-agent-657qw\" (UID: \"2781ca06-e247-4b8b-879e-72a32dbdc0bf\") " pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530249    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-kubelet\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530257    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-run-systemd\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-run-systemd\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530322    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-sysconfig\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530320    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-host\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530351    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-slash\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.531126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530350    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-var-lib-openvswitch\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530374    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-log-socket\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530387    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-cni-bin\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530387    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-var-lib-openvswitch\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530414    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-run-ovn\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530422    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-systemd-units\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530479    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf351621-f759-4dd6-8e96-0b52739329da-ovnkube-script-lib\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530535    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-etc-openvswitch\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-run-openvswitch\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530593    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-lib-modules\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530602    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-sysctl-d\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530643    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530684    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-etc-openvswitch\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530701    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-var-lib-kubelet\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530830    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-run\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530885    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/def9cc97-3b3b-455a-a095-21a14521bb12-kubelet-config\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530920    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb-host-slash\") pod \"iptables-alerter-2886f\" (UID: \"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb\") " pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.532260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-kubernetes\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.530974    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2781ca06-e247-4b8b-879e-72a32dbdc0bf-konnectivity-ca\") pod \"konnectivity-agent-657qw\" (UID: \"2781ca06-e247-4b8b-879e-72a32dbdc0bf\") " pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb-host-slash\") pod \"iptables-alerter-2886f\" (UID: \"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb\") " pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531065    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-kubernetes\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-sys\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531251    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf351621-f759-4dd6-8e96-0b52739329da-ovnkube-script-lib\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531308    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-sys\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531366    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-run-openvswitch\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf351621-f759-4dd6-8e96-0b52739329da-host-cni-bin\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531440    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf351621-f759-4dd6-8e96-0b52739329da-ovnkube-config\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531596    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-lib-modules\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531644    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-systemd\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.531647    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e047fbb7-6b0e-4ad1-8348-de32e09d26ad-tmp-dir\") pod \"node-resolver-772xx\" (UID: \"e047fbb7-6b0e-4ad1-8348-de32e09d26ad\") " pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.533430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.532583    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf351621-f759-4dd6-8e96-0b52739329da-ovn-node-metrics-cert\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.534086 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.534060    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/892e666e-1ad3-4df7-a721-7ecf591e8868-tmp\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.534506 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.534480    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2781ca06-e247-4b8b-879e-72a32dbdc0bf-agent-certs\") pod \"konnectivity-agent-657qw\" (UID: \"2781ca06-e247-4b8b-879e-72a32dbdc0bf\") " pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:15:38.536163 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.536136    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/892e666e-1ad3-4df7-a721-7ecf591e8868-etc-tuned\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.582624 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.582530    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:38.582624 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.582557    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:38.582624 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.582568    2582 projected.go:196] Error preparing data for projected volume kube-api-access-msnzf for pod openshift-network-diagnostics/network-check-target-kqszq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:38.582908 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:38.582651    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf podName:1b5ce203-55c6-4d32-92d1-a39eb10a0d2f nodeName:}" failed. No retries permitted until 2026-07-16 21:15:39.082634265 +0000 UTC m=+3.139465156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-msnzf" (UniqueName: "kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf") pod "network-check-target-kqszq" (UID: "1b5ce203-55c6-4d32-92d1-a39eb10a0d2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:38.585427 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.585397    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cmxl\" (UniqueName: \"kubernetes.io/projected/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-kube-api-access-4cmxl\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:38.585943 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.585915    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7kvt\" (UniqueName: \"kubernetes.io/projected/a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb-kube-api-access-g7kvt\") pod \"iptables-alerter-2886f\" (UID: \"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb\") " pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.586043 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.585969    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2lnl\" (UniqueName: \"kubernetes.io/projected/e047fbb7-6b0e-4ad1-8348-de32e09d26ad-kube-api-access-j2lnl\") pod \"node-resolver-772xx\" (UID: \"e047fbb7-6b0e-4ad1-8348-de32e09d26ad\") " pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.587538 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.587505    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc9r8\" (UniqueName: \"kubernetes.io/projected/892e666e-1ad3-4df7-a721-7ecf591e8868-kube-api-access-cc9r8\") pod \"tuned-xl8zf\" (UID: \"892e666e-1ad3-4df7-a721-7ecf591e8868\") " pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:38.587827 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.587807    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95wpl\" (UniqueName: \"kubernetes.io/projected/cf351621-f759-4dd6-8e96-0b52739329da-kube-api-access-95wpl\") pod \"ovnkube-node-84xhp\" (UID: \"cf351621-f759-4dd6-8e96-0b52739329da\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.618504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.618469    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w"
Jul 16 21:15:38.628476 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:38.628443    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ef19e0_985a_4e7e_b9b9_e7a3b33b9084.slice/crio-77a62c6affde9acfeb2b7a90d9c52d73e347ea5f042767bd7831d7163f4736c6 WatchSource:0}: Error finding container 77a62c6affde9acfeb2b7a90d9c52d73e347ea5f042767bd7831d7163f4736c6: Status 404 returned error can't find the container with id 77a62c6affde9acfeb2b7a90d9c52d73e347ea5f042767bd7831d7163f4736c6
Jul 16 21:15:38.632536 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.632513    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5bj9d"
Jul 16 21:15:38.643401 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.643380    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-64prw"
Jul 16 21:15:38.645754 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:38.645733    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc66b7d07_c6ac_49cf_9841_425d060e4222.slice/crio-e8d0912646c924c43e6a6be3f7592ab7463036598910fafba2cf6755e2fc5d1c WatchSource:0}: Error finding container e8d0912646c924c43e6a6be3f7592ab7463036598910fafba2cf6755e2fc5d1c: Status 404 returned error can't find the container with id e8d0912646c924c43e6a6be3f7592ab7463036598910fafba2cf6755e2fc5d1c
Jul 16 21:15:38.647322 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.647301    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:38.650650 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.650631    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzvc6"
Jul 16 21:15:38.652782 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:38.652759    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063fca31_aa29_467d_b4f2_0a81b45d9707.slice/crio-2c2df36e01514af41bfbd8bedcf4f9fa4cb8c78f483fa08a0676a9eac21bbec8 WatchSource:0}: Error finding container 2c2df36e01514af41bfbd8bedcf4f9fa4cb8c78f483fa08a0676a9eac21bbec8: Status 404 returned error can't find the container with id 2c2df36e01514af41bfbd8bedcf4f9fa4cb8c78f483fa08a0676a9eac21bbec8
Jul 16 21:15:38.658734 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.658714    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2886f"
Jul 16 21:15:38.659691 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:38.659671    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63fecfd_97f0_49e3_b741_c240ed7357ac.slice/crio-ff2d079c9540cca4beb30245637eabffc18ab40441317436622fc1b5888a495b WatchSource:0}: Error finding container ff2d079c9540cca4beb30245637eabffc18ab40441317436622fc1b5888a495b: Status 404 returned error can't find the container with id ff2d079c9540cca4beb30245637eabffc18ab40441317436622fc1b5888a495b
Jul 16 21:15:38.668038 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.668015    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:15:38.668568 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:38.668546    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43a4bfd_ed1a_46e4_b30e_467e49f0e6eb.slice/crio-d6ceb51b2083d2a98b82265d5b8a3f4735b4c0af4294b2a83da347788ba00858 WatchSource:0}: Error finding container d6ceb51b2083d2a98b82265d5b8a3f4735b4c0af4294b2a83da347788ba00858: Status 404 returned error can't find the container with id d6ceb51b2083d2a98b82265d5b8a3f4735b4c0af4294b2a83da347788ba00858
Jul 16 21:15:38.675939 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.675915    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:15:38.683515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.683487    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-772xx"
Jul 16 21:15:38.690499 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:38.690480    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xl8zf"
Jul 16 21:15:39.034376 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.034283    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:39.034376 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.034332    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:39.034590 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:39.034446    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:39.034590 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:39.034450    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:39.034590 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:39.034513    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs podName:0265e540-9cd6-45da-aa44-bc5d8a5a9a42 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:40.034493767 +0000 UTC m=+4.091324674 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs") pod "network-metrics-daemon-hlv9v" (UID: "0265e540-9cd6-45da-aa44-bc5d8a5a9a42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:39.034590 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:39.034532    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret podName:def9cc97-3b3b-455a-a095-21a14521bb12 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:40.034522161 +0000 UTC m=+4.091353057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret") pod "global-pull-secret-syncer-h98vh" (UID: "def9cc97-3b3b-455a-a095-21a14521bb12") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:39.135754 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.135712    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:39.135932 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:39.135906    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:39.135989 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:39.135943    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:39.135989 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:39.135960    2582 projected.go:196] Error preparing data for projected volume kube-api-access-msnzf for pod openshift-network-diagnostics/network-check-target-kqszq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:39.136091 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:39.136034    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf podName:1b5ce203-55c6-4d32-92d1-a39eb10a0d2f nodeName:}" failed. No retries permitted until 2026-07-16 21:15:40.136011023 +0000 UTC m=+4.192841940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-msnzf" (UniqueName: "kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf") pod "network-check-target-kqszq" (UID: "1b5ce203-55c6-4d32-92d1-a39eb10a0d2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:39.404740 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.404631    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:10:38 +0000 UTC" deadline="2028-02-01 17:55:56.322007369 +0000 UTC"
Jul 16 21:15:39.404740 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.404668    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13556h40m16.917343211s"
Jul 16 21:15:39.500820 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.500783    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:39.501034 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:39.500928    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:39.514355 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.514310    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2886f" event={"ID":"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb","Type":"ContainerStarted","Data":"d6ceb51b2083d2a98b82265d5b8a3f4735b4c0af4294b2a83da347788ba00858"}
Jul 16 21:15:39.515734 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.515705    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzvc6" event={"ID":"c63fecfd-97f0-49e3-b741-c240ed7357ac","Type":"ContainerStarted","Data":"ff2d079c9540cca4beb30245637eabffc18ab40441317436622fc1b5888a495b"}
Jul 16 21:15:39.517468 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.517438    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-64prw" event={"ID":"063fca31-aa29-467d-b4f2-0a81b45d9707","Type":"ContainerStarted","Data":"2c2df36e01514af41bfbd8bedcf4f9fa4cb8c78f483fa08a0676a9eac21bbec8"}
Jul 16 21:15:39.518677 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.518650    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5bj9d" event={"ID":"c66b7d07-c6ac-49cf-9841-425d060e4222","Type":"ContainerStarted","Data":"e8d0912646c924c43e6a6be3f7592ab7463036598910fafba2cf6755e2fc5d1c"}
Jul 16 21:15:39.520176 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:39.520148    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w" event={"ID":"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084","Type":"ContainerStarted","Data":"77a62c6affde9acfeb2b7a90d9c52d73e347ea5f042767bd7831d7163f4736c6"}
Jul 16 21:15:40.043931 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:40.043870    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:40.043931 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:40.043921    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:40.044286 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.044071    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:40.044286 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.044163    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs podName:0265e540-9cd6-45da-aa44-bc5d8a5a9a42 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:42.04414139 +0000 UTC m=+6.100972297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs") pod "network-metrics-daemon-hlv9v" (UID: "0265e540-9cd6-45da-aa44-bc5d8a5a9a42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:40.044286 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.044239    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:40.044286 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.044279    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret podName:def9cc97-3b3b-455a-a095-21a14521bb12 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:42.044266975 +0000 UTC m=+6.101097882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret") pod "global-pull-secret-syncer-h98vh" (UID: "def9cc97-3b3b-455a-a095-21a14521bb12") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:40.145374 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:40.144701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:40.145374 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.144897    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:40.145374 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.144917    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:40.145374 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.144929    2582 projected.go:196] Error preparing data for projected volume kube-api-access-msnzf for pod openshift-network-diagnostics/network-check-target-kqszq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:40.145374 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.144989    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf podName:1b5ce203-55c6-4d32-92d1-a39eb10a0d2f nodeName:}" failed. No retries permitted until 2026-07-16 21:15:42.14496957 +0000 UTC m=+6.201800487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-msnzf" (UniqueName: "kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf") pod "network-check-target-kqszq" (UID: "1b5ce203-55c6-4d32-92d1-a39eb10a0d2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:40.404916 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:40.404814    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:10:38 +0000 UTC" deadline="2028-03-11 16:11:24.637689489 +0000 UTC"
Jul 16 21:15:40.404916 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:40.404860    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14490h55m44.232834664s"
Jul 16 21:15:40.410696 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:40.410665    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf351621_f759_4dd6_8e96_0b52739329da.slice/crio-2ae3e5673e5c112206ae9e7e0cd7cd321eeccac245cbdc40eb024313c07650aa WatchSource:0}: Error finding container 2ae3e5673e5c112206ae9e7e0cd7cd321eeccac245cbdc40eb024313c07650aa: Status 404 returned error can't find the container with id 2ae3e5673e5c112206ae9e7e0cd7cd321eeccac245cbdc40eb024313c07650aa
Jul 16 21:15:40.451990 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:40.451953    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode047fbb7_6b0e_4ad1_8348_de32e09d26ad.slice/crio-0c7411cca228c13aebca12d1dbcda6c26c9a6e47b12160a8bb2d1a8084ce9973 WatchSource:0}: Error finding container 0c7411cca228c13aebca12d1dbcda6c26c9a6e47b12160a8bb2d1a8084ce9973: Status 404 returned error can't find the container with id 0c7411cca228c13aebca12d1dbcda6c26c9a6e47b12160a8bb2d1a8084ce9973
Jul 16 21:15:40.501997 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:40.501342    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:40.501997 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.501479    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:40.501997 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:40.501812    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:40.501997 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:40.501919    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:40.527111 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:40.526781    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-772xx" event={"ID":"e047fbb7-6b0e-4ad1-8348-de32e09d26ad","Type":"ContainerStarted","Data":"0c7411cca228c13aebca12d1dbcda6c26c9a6e47b12160a8bb2d1a8084ce9973"}
Jul 16 21:15:40.529341 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:40.529295    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" event={"ID":"cf351621-f759-4dd6-8e96-0b52739329da","Type":"ContainerStarted","Data":"2ae3e5673e5c112206ae9e7e0cd7cd321eeccac245cbdc40eb024313c07650aa"}
Jul 16 21:15:41.450877 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:41.448017    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892e666e_1ad3_4df7_a721_7ecf591e8868.slice/crio-4536af8aca410b4fc2e7397af7115939e3f3cb4d66cbcdbca6bf0cc497122b2d WatchSource:0}: Error finding container 4536af8aca410b4fc2e7397af7115939e3f3cb4d66cbcdbca6bf0cc497122b2d: Status 404 returned error can't find the container with id 4536af8aca410b4fc2e7397af7115939e3f3cb4d66cbcdbca6bf0cc497122b2d
Jul 16 21:15:41.500751 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:41.500716    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:41.500927 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:41.500843    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:41.532015 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:41.531978    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xl8zf" event={"ID":"892e666e-1ad3-4df7-a721-7ecf591e8868","Type":"ContainerStarted","Data":"4536af8aca410b4fc2e7397af7115939e3f3cb4d66cbcdbca6bf0cc497122b2d"}
Jul 16 21:15:42.061384 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:42.061339    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:42.061574 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:42.061392    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:42.061663 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.061574    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:42.061663 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.061638    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs podName:0265e540-9cd6-45da-aa44-bc5d8a5a9a42 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:46.061617738 +0000 UTC m=+10.118448634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs") pod "network-metrics-daemon-hlv9v" (UID: "0265e540-9cd6-45da-aa44-bc5d8a5a9a42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:42.062126 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.062083    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:42.062248 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.062140    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret podName:def9cc97-3b3b-455a-a095-21a14521bb12 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:46.062126061 +0000 UTC m=+10.118956968 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret") pod "global-pull-secret-syncer-h98vh" (UID: "def9cc97-3b3b-455a-a095-21a14521bb12") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:42.104678 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:15:42.104638    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2781ca06_e247_4b8b_879e_72a32dbdc0bf.slice/crio-99f5d2ec1b6f3a95bcf6106a5394055049ce5b734d9f87acc4e500dca46b2d70 WatchSource:0}: Error finding container 99f5d2ec1b6f3a95bcf6106a5394055049ce5b734d9f87acc4e500dca46b2d70: Status 404 returned error can't find the container with id 99f5d2ec1b6f3a95bcf6106a5394055049ce5b734d9f87acc4e500dca46b2d70
Jul 16 21:15:42.163301 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:42.162089    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:42.163301 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.162463    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:42.163301 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.162486    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:42.163301 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.162500    2582 projected.go:196] Error preparing data for projected volume kube-api-access-msnzf for pod openshift-network-diagnostics/network-check-target-kqszq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:42.163301 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.162563    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf podName:1b5ce203-55c6-4d32-92d1-a39eb10a0d2f nodeName:}" failed. No retries permitted until 2026-07-16 21:15:46.162542064 +0000 UTC m=+10.219372961 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-msnzf" (UniqueName: "kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf") pod "network-check-target-kqszq" (UID: "1b5ce203-55c6-4d32-92d1-a39eb10a0d2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:42.505070 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:42.504050    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:42.505070 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.504278    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:42.505070 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:42.504381    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:42.505070 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:42.504481    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:42.535894 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:42.535845    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-657qw" event={"ID":"2781ca06-e247-4b8b-879e-72a32dbdc0bf","Type":"ContainerStarted","Data":"99f5d2ec1b6f3a95bcf6106a5394055049ce5b734d9f87acc4e500dca46b2d70"}
Jul 16 21:15:43.500612 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:43.500520    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:43.500802 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:43.500669    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:44.502958 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:44.502862    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:44.503481 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:44.503018    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:44.503481 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:44.503203    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:44.503481 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:44.503317    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:45.501455 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.501170    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:45.501644 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:45.501520    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:45.549070 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.549023    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal" event={"ID":"33cd532730a76493a09bd77465fc98c4","Type":"ContainerStarted","Data":"0f0304941b96206becd93946453451fd637d31bd48f36e0c81148fed2fa6d01b"}
Jul 16 21:15:45.551803 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.551766    2582 generic.go:356] "Generic (PLEG): container finished" podID="f4486b65632fc154b9212b0f0a3586dc" containerID="a7becf22b093d3e6532b11997db37cbac7a2da6abd6947876d7d9a58983aecc6" exitCode=0
Jul 16 21:15:45.551962 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.551864    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal" event={"ID":"f4486b65632fc154b9212b0f0a3586dc","Type":"ContainerDied","Data":"a7becf22b093d3e6532b11997db37cbac7a2da6abd6947876d7d9a58983aecc6"}
Jul 16 21:15:45.560608 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.560451    2582 generic.go:356] "Generic (PLEG): container finished" podID="c63fecfd-97f0-49e3-b741-c240ed7357ac" containerID="d3f03c24f2afd60b718e47c07598e124250f1d1caf53bcc119dc55317eaa3535" exitCode=0
Jul 16 21:15:45.560608 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.560550    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzvc6" event={"ID":"c63fecfd-97f0-49e3-b741-c240ed7357ac","Type":"ContainerDied","Data":"d3f03c24f2afd60b718e47c07598e124250f1d1caf53bcc119dc55317eaa3535"}
Jul 16 21:15:45.564241 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.564138    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-64prw" event={"ID":"063fca31-aa29-467d-b4f2-0a81b45d9707","Type":"ContainerStarted","Data":"a4987588b28e18ec08ed9d61a1a19a4dd8c3e8e1e119ef52333190f29cfcdaa0"}
Jul 16 21:15:45.566545 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.566477    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w" event={"ID":"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084","Type":"ContainerStarted","Data":"066a1fa73bf09150d9a466437a06e52cddaf61577b76791ca0c5f0529c51d985"}
Jul 16 21:15:45.570289 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.570259    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-772xx" event={"ID":"e047fbb7-6b0e-4ad1-8348-de32e09d26ad","Type":"ContainerStarted","Data":"6602741e7d3488fbb6049e92a132ff42046394b64eb2c99d420a48fecd7c3762"}
Jul 16 21:15:45.621816 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.621745    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-201.ec2.internal" podStartSLOduration=7.621720534 podStartE2EDuration="7.621720534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:45.59551124 +0000 UTC m=+9.652342159" watchObservedRunningTime="2026-07-16 21:15:45.621720534 +0000 UTC m=+9.678551452"
Jul 16 21:15:45.712303 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.712223    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-772xx" podStartSLOduration=4.278370884 podStartE2EDuration="8.712195599s" totalImagesPullingTime="4.433824715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:45.679506359 +0000 UTC m=+9.736337289" watchObservedRunningTime="2026-07-16 21:15:45.712195599 +0000 UTC m=+9.769026520"
Jul 16 21:15:45.712800 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:45.712742    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-64prw" podStartSLOduration=2.764837525 podStartE2EDuration="8.71272725s" totalImagesPullingTime="5.947889725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:45.712288685 +0000 UTC m=+9.769119598" watchObservedRunningTime="2026-07-16 21:15:45.71272725 +0000 UTC m=+9.769558166"
Jul 16 21:15:46.099248 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:46.099201    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:46.099248 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:46.099251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:46.099532 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.099378    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:46.099532 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.099383    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:46.099532 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.099441    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs podName:0265e540-9cd6-45da-aa44-bc5d8a5a9a42 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:54.099422539 +0000 UTC m=+18.156253440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs") pod "network-metrics-daemon-hlv9v" (UID: "0265e540-9cd6-45da-aa44-bc5d8a5a9a42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:46.099532 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.099511    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret podName:def9cc97-3b3b-455a-a095-21a14521bb12 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:54.099499386 +0000 UTC m=+18.156330276 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret") pod "global-pull-secret-syncer-h98vh" (UID: "def9cc97-3b3b-455a-a095-21a14521bb12") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:46.200624 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:46.200007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:46.200624 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.200204    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:46.200624 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.200230    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:46.200624 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.200244    2582 projected.go:196] Error preparing data for projected volume kube-api-access-msnzf for pod openshift-network-diagnostics/network-check-target-kqszq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:46.200624 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.200305    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf podName:1b5ce203-55c6-4d32-92d1-a39eb10a0d2f nodeName:}" failed. No retries permitted until 2026-07-16 21:15:54.200287178 +0000 UTC m=+18.257118086 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-msnzf" (UniqueName: "kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf") pod "network-check-target-kqszq" (UID: "1b5ce203-55c6-4d32-92d1-a39eb10a0d2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:46.503614 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:46.503529    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:46.503768 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.503649    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:46.504064 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:46.504044    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:46.504192 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:46.504176    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:46.584060 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:46.584011    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal" event={"ID":"f4486b65632fc154b9212b0f0a3586dc","Type":"ContainerStarted","Data":"67b3783ff974d8bb120d68330ab8396ca156145ffea8fe26a5e6dafe2d1450f4"}
Jul 16 21:15:46.588893 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:46.588858    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2886f" event={"ID":"a43a4bfd-ed1a-46e4-b30e-467e49f0e6eb","Type":"ContainerStarted","Data":"271560996fb608c079aab4e52094c9ea757148074bad5db6f257ff81d30116dc"}
Jul 16 21:15:46.621391 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:46.621339    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-201.ec2.internal" podStartSLOduration=8.621318266 podStartE2EDuration="8.621318266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:46.620204716 +0000 UTC m=+10.677035627" watchObservedRunningTime="2026-07-16 21:15:46.621318266 +0000 UTC m=+10.678149179"
Jul 16 21:15:47.501222 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:47.501191    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:47.501405 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:47.501327    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:48.501365 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:48.501189    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:48.501365 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:48.501299    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:48.501365 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:48.501343    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:48.501893 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:48.501450    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:49.501384 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:49.501153    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:49.501819 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:49.501556    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:50.501203 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:50.501172    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:50.501203 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:50.501198    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:50.501439 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:50.501305    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:50.501439 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:50.501383    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:51.500764 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:51.500674    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:51.500919 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:51.500815    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:52.500799 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:52.500763    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:52.501323 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:52.500814    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:52.501323 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:52.500899    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:52.501323 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:52.501036    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:53.501240 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:53.501211    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:53.501762 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:53.501334    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:54.163596 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:54.163561    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:54.163596 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:54.163599    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:54.163835 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.163713    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:54.163835 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.163713    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:54.163835 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.163777    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs podName:0265e540-9cd6-45da-aa44-bc5d8a5a9a42 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:10.16376233 +0000 UTC m=+34.220593221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs") pod "network-metrics-daemon-hlv9v" (UID: "0265e540-9cd6-45da-aa44-bc5d8a5a9a42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:54.163835 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.163791    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret podName:def9cc97-3b3b-455a-a095-21a14521bb12 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:10.163785295 +0000 UTC m=+34.220616186 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret") pod "global-pull-secret-syncer-h98vh" (UID: "def9cc97-3b3b-455a-a095-21a14521bb12") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:54.264870 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:54.264630    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:54.264870 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.264792    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:54.264870 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.264815    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:54.264870 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.264828    2582 projected.go:196] Error preparing data for projected volume kube-api-access-msnzf for pod openshift-network-diagnostics/network-check-target-kqszq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:54.265317 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.265292    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf podName:1b5ce203-55c6-4d32-92d1-a39eb10a0d2f nodeName:}" failed. No retries permitted until 2026-07-16 21:16:10.265272849 +0000 UTC m=+34.322103752 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-msnzf" (UniqueName: "kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf") pod "network-check-target-kqszq" (UID: "1b5ce203-55c6-4d32-92d1-a39eb10a0d2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:54.501232 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:54.501142    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:54.501232 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:54.501159    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:54.501767 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.501270    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:54.501767 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:54.501730    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:55.500893 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:55.500862    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:55.501066 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:55.500970    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:56.501560 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:56.501516    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:56.502026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:56.501607    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:56.502026 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:56.501619    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:56.502026 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:56.501732    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:57.501189 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:57.501155    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:57.501390 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:57.501295    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:15:58.501364 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:58.501327    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:15:58.501904 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:58.501449    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:15:58.501904 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:58.501329    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:15:58.501904 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:58.501890    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:15:59.500407 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:15:59.500380    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:15:59.500566 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:15:59.500483    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:16:00.501322 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:00.501289    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:00.501820 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:00.501425    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:16:00.501820 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:00.501470    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:00.501820 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:00.501572    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:16:01.222847 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.222439    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 21:16:01.386361 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.386274    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T21:16:01.222639981Z","UUID":"ffb51ff3-db6e-4f4c-9860-52e5b60db392","Handler":null,"Name":"","Endpoint":""}
Jul 16 21:16:01.388357 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.388329    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 21:16:01.388463 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.388377    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 21:16:01.500875 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.500853    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:01.500970 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:01.500947    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:16:01.620890 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.620867    2582 generic.go:356] "Generic (PLEG): container finished" podID="c63fecfd-97f0-49e3-b741-c240ed7357ac" containerID="fec20651b0e2c4ad1ad04c0043f113809ea198d6bbaa51ebd0330516c0ccc2c0" exitCode=0
Jul 16 21:16:01.621540 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.620951    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzvc6" event={"ID":"c63fecfd-97f0-49e3-b741-c240ed7357ac","Type":"ContainerDied","Data":"fec20651b0e2c4ad1ad04c0043f113809ea198d6bbaa51ebd0330516c0ccc2c0"}
Jul 16 21:16:01.622574 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.622551    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5bj9d" event={"ID":"c66b7d07-c6ac-49cf-9841-425d060e4222","Type":"ContainerStarted","Data":"aa67980fb67e7d9a805560365134b04fd123d72e9a06383a2774c9a9020ff133"}
Jul 16 21:16:01.624395 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.624375    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w" event={"ID":"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084","Type":"ContainerStarted","Data":"a4e8e4e52a8f608e7b4ec708c3f91a9382f03b44aa3d456f12768fa83b399aea"}
Jul 16 21:16:01.626970 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.626949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" event={"ID":"cf351621-f759-4dd6-8e96-0b52739329da","Type":"ContainerStarted","Data":"f52d671b49c540c960c51fa86d5ff904c55eff1c79b62bf3b430c2a664f730da"}
Jul 16 21:16:01.627053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.627010    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" event={"ID":"cf351621-f759-4dd6-8e96-0b52739329da","Type":"ContainerStarted","Data":"1ca9b17da73cbb4ca73c894eb639fd3a3f631850a25e46129d52a7beb694718d"}
Jul 16 21:16:01.627053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.627040    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" event={"ID":"cf351621-f759-4dd6-8e96-0b52739329da","Type":"ContainerStarted","Data":"34dbea1b5ec95aec318fcd258ebcf09754a7c2b61f46ed33b26ee3620aac3d7c"}
Jul 16 21:16:01.627167 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.627110    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" event={"ID":"cf351621-f759-4dd6-8e96-0b52739329da","Type":"ContainerStarted","Data":"a268dbbc8bb4aee7164be2f2f1d92462af7c713523a515104594ac0e5b5f65e4"}
Jul 16 21:16:01.627167 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.627142    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" event={"ID":"cf351621-f759-4dd6-8e96-0b52739329da","Type":"ContainerStarted","Data":"b81a3515610e69903345c7466486cfb34334620ec63c0b54bc6b21a9d1b5f313"}
Jul 16 21:16:01.628338 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.628316    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xl8zf" event={"ID":"892e666e-1ad3-4df7-a721-7ecf591e8868","Type":"ContainerStarted","Data":"629aed0a5d254e28ac20d2f26a14b06dfa253b17590a6b01e627450e07160579"}
Jul 16 21:16:01.629437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.629418    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-657qw" event={"ID":"2781ca06-e247-4b8b-879e-72a32dbdc0bf","Type":"ContainerStarted","Data":"1dc9dcb44111f56b5389bc57d7a7eabdeaf5414d7e70c8c3760fbefec8f5b330"}
Jul 16 21:16:01.649498 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.649461    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2886f" podStartSLOduration=18.709797767 podStartE2EDuration="24.649449677s" totalImagesPullingTime="5.93965191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:46.644385761 +0000 UTC m=+10.701216675" watchObservedRunningTime="2026-07-16 21:16:01.649449677 +0000 UTC m=+25.706280592"
Jul 16 21:16:01.679277 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.679229    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5bj9d" podStartSLOduration=2.232539694 podStartE2EDuration="24.679213775s" totalImagesPullingTime="22.446674081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:01.679072347 +0000 UTC m=+25.735903321" watchObservedRunningTime="2026-07-16 21:16:01.679213775 +0000 UTC m=+25.736044687"
Jul 16 21:16:01.735555 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:01.735361    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-657qw" podStartSLOduration=5.795120328 podStartE2EDuration="24.735344884s" totalImagesPullingTime="18.940224556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:01.709210135 +0000 UTC m=+25.766041072" watchObservedRunningTime="2026-07-16 21:16:01.735344884 +0000 UTC m=+25.792175796"
Jul 16 21:16:02.501392 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:02.501362    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:02.501392 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:02.501400    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:02.501639 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:02.501503    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:16:02.501639 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:02.501603    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:16:02.633860 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:02.633787    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w" event={"ID":"98ef19e0-985a-4e7e-b9b9-e7a3b33b9084","Type":"ContainerStarted","Data":"6fd2667de0ed942187ba354de3e9d0af5aa48a83be52d392e457b15d60e6564f"}
Jul 16 21:16:02.636507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:02.636475    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" event={"ID":"cf351621-f759-4dd6-8e96-0b52739329da","Type":"ContainerStarted","Data":"620ba46bf7c922d52ca1e01e802a8f81e6ec081edfe338dbc78f887adabc0f34"}
Jul 16 21:16:02.664248 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:02.664207    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bdh2w" podStartSLOduration=2.632331234 podStartE2EDuration="25.664194576s" totalImagesPullingTime="23.031863342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:02.664172947 +0000 UTC m=+26.721003859" watchObservedRunningTime="2026-07-16 21:16:02.664194576 +0000 UTC m=+26.721025489"
Jul 16 21:16:02.664350 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:02.664299    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xl8zf" podStartSLOduration=6.064019325 podStartE2EDuration="25.664295335s" totalImagesPullingTime="19.60027601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:01.73597378 +0000 UTC m=+25.792804671" watchObservedRunningTime="2026-07-16 21:16:02.664295335 +0000 UTC m=+26.721126247"
Jul 16 21:16:03.500603 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:03.500563    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:03.500911 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:03.500698    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:16:03.640451 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:03.640418    2582 generic.go:356] "Generic (PLEG): container finished" podID="c63fecfd-97f0-49e3-b741-c240ed7357ac" containerID="e264252ef966e9cee6d191abd73a7d518b1091cf35a0071660397b44ab101614" exitCode=0
Jul 16 21:16:03.640920 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:03.640501    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzvc6" event={"ID":"c63fecfd-97f0-49e3-b741-c240ed7357ac","Type":"ContainerDied","Data":"e264252ef966e9cee6d191abd73a7d518b1091cf35a0071660397b44ab101614"}
Jul 16 21:16:04.172023 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.171991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:04.280448 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.280408    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:04.370840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.370795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:04.501146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.501045    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:04.501313 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.501056    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:04.501313 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:04.501166    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:16:04.501313 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:04.501243    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:16:04.581725 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.581679    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:04.645887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.645856    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" event={"ID":"cf351621-f759-4dd6-8e96-0b52739329da","Type":"ContainerStarted","Data":"ad549958397f110b931a926b082fb7e36b4d61a1cdf03c08be6b1c7f86198d2b"}
Jul 16 21:16:04.694162 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.694145    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:04.797200 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.797177    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:04.902140 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:04.902116    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:05.172524 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.172439    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:05.269481 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.269456    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:16:05.270105 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.270051    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:16:05.411818 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.411782    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:05.500697 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.500628    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:05.500843 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:05.500718    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:16:05.534715 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.534690    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:05.650641 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.650607    2582 generic.go:356] "Generic (PLEG): container finished" podID="c63fecfd-97f0-49e3-b741-c240ed7357ac" containerID="8bd6faf8986ad6a56060d396784537b08d5e25d86cea34cbaf489591d0939499" exitCode=0
Jul 16 21:16:05.651024 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.650687    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzvc6" event={"ID":"c63fecfd-97f0-49e3-b741-c240ed7357ac","Type":"ContainerDied","Data":"8bd6faf8986ad6a56060d396784537b08d5e25d86cea34cbaf489591d0939499"}
Jul 16 21:16:05.651024 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.650894    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:16:05.651413 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:05.651397    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-657qw"
Jul 16 21:16:06.204868 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:06.204695    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:06.307582 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:06.307548    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:06.501620 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:06.501583    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:06.501821 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:06.501682    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:06.501821 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:06.501690    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:16:06.501821 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:06.501800    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:16:06.658445 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:06.658347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" event={"ID":"cf351621-f759-4dd6-8e96-0b52739329da","Type":"ContainerStarted","Data":"097d4926046a58beea7bc0505a22239457d370ca11473e6168416b5af29821e3"}
Jul 16 21:16:06.709730 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:06.709680    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp" podStartSLOduration=9.088438025 podStartE2EDuration="29.709665014s" totalImagesPullingTime="20.621226989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:06.709321841 +0000 UTC m=+30.766152752" watchObservedRunningTime="2026-07-16 21:16:06.709665014 +0000 UTC m=+30.766495925"
Jul 16 21:16:07.501406 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:07.501367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:07.501609 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:07.501500    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:16:07.527912 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:07.527876    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:07.661130 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:07.661060    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:16:07.661567 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:07.661184    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:16:07.661567 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:07.661235    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:16:07.678745 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:07.678721    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:16:07.678902 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:07.678843    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:16:07.742709 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:07.742675    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:07.926245 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:07.926203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:08.187522 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:08.187440    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:08.288218 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:08.288187    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:08.393473 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:08.393442    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:08.505343 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:08.505268    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:08.505505 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:08.505381    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:16:08.505505 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:08.505439    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:08.505609 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:08.505558    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:16:09.501537 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:09.501510    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:09.501892 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:09.501642    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:16:09.733118 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:09.733018    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h98vh"]
Jul 16 21:16:09.733273 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:09.733151    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:09.733273 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:09.733251    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:16:09.754332 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:09.754305    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kqszq"]
Jul 16 21:16:09.754512 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:09.754401    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:09.754512 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:09.754478    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:16:09.754936 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:09.754891    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hlv9v"]
Jul 16 21:16:09.755109 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:09.754971    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:09.755109 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:09.755059    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:16:10.183095 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:10.183038    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:10.183257 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:10.183110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:10.183257 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:10.183194    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:16:10.183347 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:10.183275    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret podName:def9cc97-3b3b-455a-a095-21a14521bb12 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:42.183254375 +0000 UTC m=+66.240085269 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret") pod "global-pull-secret-syncer-h98vh" (UID: "def9cc97-3b3b-455a-a095-21a14521bb12") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:16:10.183347 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:10.183273    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:16:10.183347 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:10.183342    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs podName:0265e540-9cd6-45da-aa44-bc5d8a5a9a42 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:42.183323633 +0000 UTC m=+66.240154530 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs") pod "network-metrics-daemon-hlv9v" (UID: "0265e540-9cd6-45da-aa44-bc5d8a5a9a42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:16:10.284270 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:10.284230    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:10.284448 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:10.284428    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:16:10.284496 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:10.284455    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:16:10.284496 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:10.284466    2582 projected.go:196] Error preparing data for projected volume kube-api-access-msnzf for pod openshift-network-diagnostics/network-check-target-kqszq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:16:10.284586 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:10.284518    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf podName:1b5ce203-55c6-4d32-92d1-a39eb10a0d2f nodeName:}" failed. No retries permitted until 2026-07-16 21:16:42.2845054 +0000 UTC m=+66.341336294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-msnzf" (UniqueName: "kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf") pod "network-check-target-kqszq" (UID: "1b5ce203-55c6-4d32-92d1-a39eb10a0d2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:16:10.669322 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:10.669129    2582 generic.go:356] "Generic (PLEG): container finished" podID="c63fecfd-97f0-49e3-b741-c240ed7357ac" containerID="fec047cb2cf87dee6a05a29d4b5a22b2bc7572df679f7d3d724636416590b155" exitCode=0
Jul 16 21:16:10.669322 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:10.669203    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzvc6" event={"ID":"c63fecfd-97f0-49e3-b741-c240ed7357ac","Type":"ContainerDied","Data":"fec047cb2cf87dee6a05a29d4b5a22b2bc7572df679f7d3d724636416590b155"}
Jul 16 21:16:11.501234 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:11.501199    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:11.501453 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:11.501308    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:11.501453 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:11.501402    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:16:11.501453 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:11.501309    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:16:11.501453 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:11.501312    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:11.501611 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:11.501489    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:16:11.675237 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:11.675203    2582 generic.go:356] "Generic (PLEG): container finished" podID="c63fecfd-97f0-49e3-b741-c240ed7357ac" containerID="40c12530d8402dd67825afacff57afec4827e466f0bbe6190e1a0488e8b63a4b" exitCode=0
Jul 16 21:16:11.675609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:11.675263    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzvc6" event={"ID":"c63fecfd-97f0-49e3-b741-c240ed7357ac","Type":"ContainerDied","Data":"40c12530d8402dd67825afacff57afec4827e466f0bbe6190e1a0488e8b63a4b"}
Jul 16 21:16:11.710341 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:11.710310    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:11.912861 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:11.912831    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:12.121007 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:12.120974    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:12.288108 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:12.288062    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:12.452832 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:12.452750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:12.680634 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:12.680596    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzvc6" event={"ID":"c63fecfd-97f0-49e3-b741-c240ed7357ac","Type":"ContainerStarted","Data":"a5b034d37d4e6c4956d9c56b82a5ad9dbde9009e30e3c933aabe54fe6772a11d"}
Jul 16 21:16:12.965290 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:12.965259    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:13.057746 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.057716    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:13.158413 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.158379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:13.500872 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.500838    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:13.500872 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.500863    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:13.501120 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.500897    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:13.501120 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:13.501005    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hlv9v" podUID="0265e540-9cd6-45da-aa44-bc5d8a5a9a42"
Jul 16 21:16:13.501120 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:13.501040    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kqszq" podUID="1b5ce203-55c6-4d32-92d1-a39eb10a0d2f"
Jul 16 21:16:13.501303 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:16:13.501140    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h98vh" podUID="def9cc97-3b3b-455a-a095-21a14521bb12"
Jul 16 21:16:13.805799 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.805773    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-201.ec2.internal" event="NodeReady"
Jul 16 21:16:13.806378 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.805914    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 21:16:13.875045 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.874991    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xzvc6" podStartSLOduration=8.278605328 podStartE2EDuration="36.874974438s" totalImagesPullingTime="28.59636911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:12.770796268 +0000 UTC m=+36.827627180" watchObservedRunningTime="2026-07-16 21:16:13.874974438 +0000 UTC m=+37.931805350"
Jul 16 21:16:13.875720 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.875648    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85fbb945c6-fdbjg"]
Jul 16 21:16:13.880314 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.880293    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:13.888208 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.888191    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:13.893320 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.893301    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 16 21:16:13.894020 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.894000    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 16 21:16:13.894289 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.894271    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 16 21:16:13.894370 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.894332    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p4mzm\""
Jul 16 21:16:13.922404 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.922374    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m25k7"]
Jul 16 21:16:13.923215 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.923198    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 16 21:16:13.931175 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.931158    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m25k7"
Jul 16 21:16:13.934521 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.934501    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 21:16:13.934810 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.934795    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 21:16:13.936459 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.936439    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85fbb945c6-fdbjg"]
Jul 16 21:16:13.936875 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.936857    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-blkm2\""
Jul 16 21:16:13.937770 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.937754    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 21:16:13.994484 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:13.994440    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m25k7"]
Jul 16 21:16:14.012143 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012114    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dd9a270b-9a17-49ca-8ae0-11304f71184f-installation-pull-secrets\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.012308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012149    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4-cert\") pod \"ingress-canary-m25k7\" (UID: \"c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4\") " pod="openshift-ingress-canary/ingress-canary-m25k7"
Jul 16 21:16:14.012308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012176    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dd9a270b-9a17-49ca-8ae0-11304f71184f-registry-certificates\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.012308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012205    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgsgr\" (UniqueName: \"kubernetes.io/projected/dd9a270b-9a17-49ca-8ae0-11304f71184f-kube-api-access-mgsgr\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.012515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012310    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dd9a270b-9a17-49ca-8ae0-11304f71184f-image-registry-private-configuration\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.012515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012349    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dd9a270b-9a17-49ca-8ae0-11304f71184f-ca-trust-extracted\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.012515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012401    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd9a270b-9a17-49ca-8ae0-11304f71184f-trusted-ca\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.012515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012435    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgwzd\" (UniqueName: \"kubernetes.io/projected/c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4-kube-api-access-wgwzd\") pod \"ingress-canary-m25k7\" (UID: \"c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4\") " pod="openshift-ingress-canary/ingress-canary-m25k7"
Jul 16 21:16:14.012515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012459    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dd9a270b-9a17-49ca-8ae0-11304f71184f-registry-tls\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.012515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.012491    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd9a270b-9a17-49ca-8ae0-11304f71184f-bound-sa-token\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.013437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.013409    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b6c6bb4db-8cb84"]
Jul 16 21:16:14.024293 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.024266    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:14.028710 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.028691    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.029985 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.029965    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pklzw"]
Jul 16 21:16:14.040931 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.040912    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.056811 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.056748    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 21:16:14.056811 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.056776    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 21:16:14.056811 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.056748    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 21:16:14.056811 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.056793    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 21:16:14.057055 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.056849    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 21:16:14.057170 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.057153    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 21:16:14.057220 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.057180    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 21:16:14.057220 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.057202    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m9h62\""
Jul 16 21:16:14.057220 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.057182    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ht7k4\""
Jul 16 21:16:14.057365 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.057223    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 21:16:14.057365 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.057309    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 21:16:14.057609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.057593    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 21:16:14.057689 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.057635    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 21:16:14.058730 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.058712    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gwgdk"]
Jul 16 21:16:14.072474 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.072444    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b6c6bb4db-8cb84"]
Jul 16 21:16:14.072603 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.072536    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.083127 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.083110    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 21:16:14.084999 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.084923    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pklzw"]
Jul 16 21:16:14.086345 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.086319    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dhx8d\""
Jul 16 21:16:14.087731 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.087706    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gwgdk"]
Jul 16 21:16:14.092421 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.092404    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 21:16:14.112968 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.112942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dd9a270b-9a17-49ca-8ae0-11304f71184f-image-registry-private-configuration\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.112968 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.112972    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dd9a270b-9a17-49ca-8ae0-11304f71184f-ca-trust-extracted\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.113148 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.112994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-serving-cert\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.113148 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113019    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-config\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.113148 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113066    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd9a270b-9a17-49ca-8ae0-11304f71184f-trusted-ca\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.113406 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-oauth-config\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.113406 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113260    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgwzd\" (UniqueName: \"kubernetes.io/projected/c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4-kube-api-access-wgwzd\") pod \"ingress-canary-m25k7\" (UID: \"c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4\") " pod="openshift-ingress-canary/ingress-canary-m25k7"
Jul 16 21:16:14.113406 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113286    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dd9a270b-9a17-49ca-8ae0-11304f71184f-registry-tls\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.113406 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113313    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-oauth-serving-cert\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.113579 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113416    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dd9a270b-9a17-49ca-8ae0-11304f71184f-ca-trust-extracted\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.113579 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113429    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd9a270b-9a17-49ca-8ae0-11304f71184f-bound-sa-token\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.113579 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113501    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dd9a270b-9a17-49ca-8ae0-11304f71184f-installation-pull-secrets\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.113579 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113531    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4-cert\") pod \"ingress-canary-m25k7\" (UID: \"c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4\") " pod="openshift-ingress-canary/ingress-canary-m25k7"
Jul 16 21:16:14.113579 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwpkk\" (UniqueName: \"kubernetes.io/projected/69590fa3-f855-4cb8-b04e-e791bafe13ec-kube-api-access-pwpkk\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.113823 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dd9a270b-9a17-49ca-8ae0-11304f71184f-registry-certificates\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.113823 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113761    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-service-ca\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.113926 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.113828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgsgr\" (UniqueName: \"kubernetes.io/projected/dd9a270b-9a17-49ca-8ae0-11304f71184f-kube-api-access-mgsgr\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.114202 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.114180    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd9a270b-9a17-49ca-8ae0-11304f71184f-trusted-ca\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.117554 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.117530    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4-cert\") pod \"ingress-canary-m25k7\" (UID: \"c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4\") " pod="openshift-ingress-canary/ingress-canary-m25k7"
Jul 16 21:16:14.117645 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.117560    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dd9a270b-9a17-49ca-8ae0-11304f71184f-installation-pull-secrets\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.117645 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.117634    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dd9a270b-9a17-49ca-8ae0-11304f71184f-image-registry-private-configuration\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.117714 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.117651    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dd9a270b-9a17-49ca-8ae0-11304f71184f-registry-tls\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.119527 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.119507    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dd9a270b-9a17-49ca-8ae0-11304f71184f-registry-certificates\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.135888 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.135862    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd9a270b-9a17-49ca-8ae0-11304f71184f-bound-sa-token\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.137321 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.137292    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgsgr\" (UniqueName: \"kubernetes.io/projected/dd9a270b-9a17-49ca-8ae0-11304f71184f-kube-api-access-mgsgr\") pod \"image-registry-85fbb945c6-fdbjg\" (UID: \"dd9a270b-9a17-49ca-8ae0-11304f71184f\") " pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.142382 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.142358    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgwzd\" (UniqueName: \"kubernetes.io/projected/c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4-kube-api-access-wgwzd\") pod \"ingress-canary-m25k7\" (UID: \"c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4\") " pod="openshift-ingress-canary/ingress-canary-m25k7"
Jul 16 21:16:14.149320 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.149297    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:14.197177 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.197140    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.214581 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ca933aa-7524-4ca8-994b-b0a96e693952-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.214653 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214593    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwpkk\" (UniqueName: \"kubernetes.io/projected/69590fa3-f855-4cb8-b04e-e791bafe13ec-kube-api-access-pwpkk\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.214653 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214615    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzlvr\" (UniqueName: \"kubernetes.io/projected/9ca933aa-7524-4ca8-994b-b0a96e693952-kube-api-access-fzlvr\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.214738 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-service-ca\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.214738 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214678    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9ca933aa-7524-4ca8-994b-b0a96e693952-crio-socket\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.214738 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214695    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ca933aa-7524-4ca8-994b-b0a96e693952-data-volume\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.214738 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214721    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f791e45e-b730-4183-9b33-70be21cff13a-config-volume\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.214924 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214744    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bkqk\" (UniqueName: \"kubernetes.io/projected/f791e45e-b730-4183-9b33-70be21cff13a-kube-api-access-8bkqk\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.214924 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-serving-cert\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.214924 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-config\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.214924 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-oauth-config\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.214924 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214906    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f791e45e-b730-4183-9b33-70be21cff13a-tmp-dir\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.215180 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214932    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ca933aa-7524-4ca8-994b-b0a96e693952-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.215180 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214972    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-oauth-serving-cert\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.215180 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.214991    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f791e45e-b730-4183-9b33-70be21cff13a-metrics-tls\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.215458 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.215441    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-service-ca\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.215560 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.215502    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-config\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.215602 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.215581    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-oauth-serving-cert\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.217939 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.217921    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-serving-cert\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.218014 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.217984    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-oauth-config\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.237523 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.237495    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwpkk\" (UniqueName: \"kubernetes.io/projected/69590fa3-f855-4cb8-b04e-e791bafe13ec-kube-api-access-pwpkk\") pod \"console-b6c6bb4db-8cb84\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") " pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.239241 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.239224    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m25k7"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.318310    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzlvr\" (UniqueName: \"kubernetes.io/projected/9ca933aa-7524-4ca8-994b-b0a96e693952-kube-api-access-fzlvr\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.318558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9ca933aa-7524-4ca8-994b-b0a96e693952-crio-socket\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.318584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ca933aa-7524-4ca8-994b-b0a96e693952-data-volume\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.318628    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f791e45e-b730-4183-9b33-70be21cff13a-config-volume\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.318652    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bkqk\" (UniqueName: \"kubernetes.io/projected/f791e45e-b730-4183-9b33-70be21cff13a-kube-api-access-8bkqk\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.318717    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f791e45e-b730-4183-9b33-70be21cff13a-tmp-dir\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.318743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ca933aa-7524-4ca8-994b-b0a96e693952-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.318776    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f791e45e-b730-4183-9b33-70be21cff13a-metrics-tls\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.318841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ca933aa-7524-4ca8-994b-b0a96e693952-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.319674    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f791e45e-b730-4183-9b33-70be21cff13a-config-volume\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.320222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9ca933aa-7524-4ca8-994b-b0a96e693952-crio-socket\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.320514    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ca933aa-7524-4ca8-994b-b0a96e693952-data-volume\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.320642    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f791e45e-b730-4183-9b33-70be21cff13a-tmp-dir\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.322059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.320993    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ca933aa-7524-4ca8-994b-b0a96e693952-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.329942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.323545    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:14.329942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.328032    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f791e45e-b730-4183-9b33-70be21cff13a-metrics-tls\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.333426 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.333379    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ca933aa-7524-4ca8-994b-b0a96e693952-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.338927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.338019    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:14.341605 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.341569    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzlvr\" (UniqueName: \"kubernetes.io/projected/9ca933aa-7524-4ca8-994b-b0a96e693952-kube-api-access-fzlvr\") pod \"insights-runtime-extractor-pklzw\" (UID: \"9ca933aa-7524-4ca8-994b-b0a96e693952\") " pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.341806 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.341782    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bkqk\" (UniqueName: \"kubernetes.io/projected/f791e45e-b730-4183-9b33-70be21cff13a-kube-api-access-8bkqk\") pod \"dns-default-gwgdk\" (UID: \"f791e45e-b730-4183-9b33-70be21cff13a\") " pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.348689 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.348666    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pklzw"
Jul 16 21:16:14.381372 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.381279    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:14.430900 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.429739    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85fbb945c6-fdbjg"]
Jul 16 21:16:14.430900 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.430566    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m25k7"]
Jul 16 21:16:14.435757 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:14.435041    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9e92ac7_c0b5_4c0f_8249_b4140a74d1e4.slice/crio-7d441003abb9170f8343867dd08ac40bc54893b016b174e7f61c77ff2488630a WatchSource:0}: Error finding container 7d441003abb9170f8343867dd08ac40bc54893b016b174e7f61c77ff2488630a: Status 404 returned error can't find the container with id 7d441003abb9170f8343867dd08ac40bc54893b016b174e7f61c77ff2488630a
Jul 16 21:16:14.471557 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.471483    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:14.549761 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.549731    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b6c6bb4db-8cb84"]
Jul 16 21:16:14.573941 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.573881    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pklzw"]
Jul 16 21:16:14.584846 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:14.584823    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca933aa_7524_4ca8_994b_b0a96e693952.slice/crio-1da2db36bc420913ffc9d74ea3bb1de77a6c8ceee571a12820b7cb5dfc001da7 WatchSource:0}: Error finding container 1da2db36bc420913ffc9d74ea3bb1de77a6c8ceee571a12820b7cb5dfc001da7: Status 404 returned error can't find the container with id 1da2db36bc420913ffc9d74ea3bb1de77a6c8ceee571a12820b7cb5dfc001da7
Jul 16 21:16:14.585946 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:14.585923    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69590fa3_f855_4cb8_b04e_e791bafe13ec.slice/crio-0af22b1195b474b48834e7bd4eb210ba02553a97538908158fed983daf6cc2a5 WatchSource:0}: Error finding container 0af22b1195b474b48834e7bd4eb210ba02553a97538908158fed983daf6cc2a5: Status 404 returned error can't find the container with id 0af22b1195b474b48834e7bd4eb210ba02553a97538908158fed983daf6cc2a5
Jul 16 21:16:14.592941 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:14.592912    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf791e45e_b730_4183_9b33_70be21cff13a.slice/crio-619d9f89799d2b08cd2e79dfad1a159f1dfb7028fbb18a944783cf175ba1c5d9 WatchSource:0}: Error finding container 619d9f89799d2b08cd2e79dfad1a159f1dfb7028fbb18a944783cf175ba1c5d9: Status 404 returned error can't find the container with id 619d9f89799d2b08cd2e79dfad1a159f1dfb7028fbb18a944783cf175ba1c5d9
Jul 16 21:16:14.593818 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.593799    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gwgdk"]
Jul 16 21:16:14.686686 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.686655    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg" event={"ID":"dd9a270b-9a17-49ca-8ae0-11304f71184f","Type":"ContainerStarted","Data":"7ff259e02ea40feeb8950e1c91103e724b74763719caf9b206e43348452ebc19"}
Jul 16 21:16:14.686816 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.686698    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg" event={"ID":"dd9a270b-9a17-49ca-8ae0-11304f71184f","Type":"ContainerStarted","Data":"9cc8b55d05a4f2b708e193a1d52c3bef84d32cedd63e73ce2c937b9b6672b7d8"}
Jul 16 21:16:14.686816 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.686725    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:14.687870 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.687840    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6c6bb4db-8cb84" event={"ID":"69590fa3-f855-4cb8-b04e-e791bafe13ec","Type":"ContainerStarted","Data":"0af22b1195b474b48834e7bd4eb210ba02553a97538908158fed983daf6cc2a5"}
Jul 16 21:16:14.688878 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.688855    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m25k7" event={"ID":"c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4","Type":"ContainerStarted","Data":"7d441003abb9170f8343867dd08ac40bc54893b016b174e7f61c77ff2488630a"}
Jul 16 21:16:14.689927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.689908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gwgdk" event={"ID":"f791e45e-b730-4183-9b33-70be21cff13a","Type":"ContainerStarted","Data":"619d9f89799d2b08cd2e79dfad1a159f1dfb7028fbb18a944783cf175ba1c5d9"}
Jul 16 21:16:14.690862 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.690843    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pklzw" event={"ID":"9ca933aa-7524-4ca8-994b-b0a96e693952","Type":"ContainerStarted","Data":"1da2db36bc420913ffc9d74ea3bb1de77a6c8ceee571a12820b7cb5dfc001da7"}
Jul 16 21:16:14.952096 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:14.952000    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:15.083768 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.083693    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:15.503333 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.501145    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:15.503333 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.501614    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:15.503333 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.502117    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:15.509220 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.507929    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 21:16:15.509220 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.508238    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 21:16:15.509220 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.508476    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 21:16:15.518204 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.517272    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tpjps\""
Jul 16 21:16:15.518390 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.518372    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 21:16:15.527770 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:15.527752    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzz4k\""
Jul 16 21:16:16.469822 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:16.469783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:16.570094 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:16.569860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:19.402538 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:19.402502    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:19.479805 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:19.479736    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:20.605733 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:20.605695    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:20.677002 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:20.676965    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:22.997065 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:22.997037    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:23.092708 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.092682    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:23.557547 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.557510    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:23.721360 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.721327    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gwgdk" event={"ID":"f791e45e-b730-4183-9b33-70be21cff13a","Type":"ContainerStarted","Data":"fd4b6fc114923c7ef91eccfb7adac3553f46f9258f69ff5631dbc5aeae362298"}
Jul 16 21:16:23.721528 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.721382    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gwgdk" event={"ID":"f791e45e-b730-4183-9b33-70be21cff13a","Type":"ContainerStarted","Data":"34f6ea11297797fcabcb8ec921e0e955d6a806e80f02e5d9a9de1da2d4a6a9dc"}
Jul 16 21:16:23.721528 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.721466    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:23.722960 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.722934    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pklzw" event={"ID":"9ca933aa-7524-4ca8-994b-b0a96e693952","Type":"ContainerStarted","Data":"bd079bf12ed529f3c7b7c81ebb80e5e32a6621030408b2cabb136dd52146375c"}
Jul 16 21:16:23.724282 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.724262    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6c6bb4db-8cb84" event={"ID":"69590fa3-f855-4cb8-b04e-e791bafe13ec","Type":"ContainerStarted","Data":"4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c"}
Jul 16 21:16:23.725629 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.725608    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m25k7" event={"ID":"c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4","Type":"ContainerStarted","Data":"dbb05a1619ab6ed68a8ff13afa07a97d6140552bbffc2cd9b01c0f190dfc634a"}
Jul 16 21:16:23.749935 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.749867    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg" podStartSLOduration=16.749854082 podStartE2EDuration="16.749854082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:14.776040349 +0000 UTC m=+38.832871261" watchObservedRunningTime="2026-07-16 21:16:23.749854082 +0000 UTC m=+47.806684994"
Jul 16 21:16:23.784014 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.783956    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b6c6bb4db-8cb84" podStartSLOduration=2.6408420550000002 podStartE2EDuration="10.783940027s" totalImagesPullingTime="8.143097972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:23.783515404 +0000 UTC m=+47.840346321" watchObservedRunningTime="2026-07-16 21:16:23.783940027 +0000 UTC m=+47.840770939"
Jul 16 21:16:23.784204 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:23.784100    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gwgdk" podStartSLOduration=2.6392483970000002 podStartE2EDuration="10.784070216s" totalImagesPullingTime="8.144821819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:23.750149375 +0000 UTC m=+47.806980288" watchObservedRunningTime="2026-07-16 21:16:23.784070216 +0000 UTC m=+47.840901128"
Jul 16 21:16:24.311598 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:24.311526    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:24.339699 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:24.339665    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:24.339882 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:24.339725    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:24.346965 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:24.346941    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:24.418625 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:24.418592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:24.457790 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:24.457737    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m25k7" podStartSLOduration=3.155849422 podStartE2EDuration="11.457720645s" totalImagesPullingTime="8.301871223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:23.824718068 +0000 UTC m=+47.881548980" watchObservedRunningTime="2026-07-16 21:16:24.457720645 +0000 UTC m=+48.514551534"
Jul 16 21:16:24.733657 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:24.733570    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:16:25.449072 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:25.449021    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:25.541760 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:25.541700    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:27.742875 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:27.742785    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pklzw" event={"ID":"9ca933aa-7524-4ca8-994b-b0a96e693952","Type":"ContainerStarted","Data":"1c98676e81d9663da0c947ec770281a2824248953c6bfe90b4da4891d6adc53e"}
Jul 16 21:16:30.182099 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:30.182043    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:30.961110 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:30.961065    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:31.170001 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:31.169960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:31.276545 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:31.276517    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:31.756362 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:31.756281    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pklzw" event={"ID":"9ca933aa-7524-4ca8-994b-b0a96e693952","Type":"ContainerStarted","Data":"c3113921b375ccd8c6bbc6d0b9b8c6364be4f8cea741a960ee65a055eb18ef6f"}
Jul 16 21:16:31.788721 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:31.788670    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pklzw" podStartSLOduration=2.609021181 podStartE2EDuration="18.788653849s" totalImagesPullingTime="16.179632668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:16:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:31.788279118 +0000 UTC m=+55.845110034" watchObservedRunningTime="2026-07-16 21:16:31.788653849 +0000 UTC m=+55.845484762"
Jul 16 21:16:32.236352 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:32.236315    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:32.357779 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:32.357745    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:32.732602 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:32.732567    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:32.855878 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:32.855808    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:33.082174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.082140    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d8f74"]
Jul 16 21:16:33.084381 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.084364    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.093686 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.093666    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 21:16:33.093794 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.093673    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 21:16:33.094522 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.094507    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 21:16:33.094638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.094621    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 21:16:33.094677 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.094659    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 21:16:33.094741 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.094722    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pnkrk\""
Jul 16 21:16:33.098772 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.098754    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 21:16:33.142566 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.142543    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:33.175808 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.175779    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a1ce953-a8b2-476c-863c-049f522071bf-sys\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.175939 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.175818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-tls\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.175939 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.175867    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a1ce953-a8b2-476c-863c-049f522071bf-metrics-client-ca\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.175939 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.175900    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-textfile\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.175939 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.175926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a1ce953-a8b2-476c-863c-049f522071bf-root\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.176158 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.175963    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdmjs\" (UniqueName: \"kubernetes.io/projected/5a1ce953-a8b2-476c-863c-049f522071bf-kube-api-access-wdmjs\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.176158 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.175998    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.176158 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.176030    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.176158 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.176093    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-wtmp\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.276704 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.276668    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmjs\" (UniqueName: \"kubernetes.io/projected/5a1ce953-a8b2-476c-863c-049f522071bf-kube-api-access-wdmjs\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.276704 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.276708    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.276938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.276729    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.276938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.276754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-wtmp\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.276938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.276804    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a1ce953-a8b2-476c-863c-049f522071bf-sys\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.276938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.276826    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-tls\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.276938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.276857    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a1ce953-a8b2-476c-863c-049f522071bf-metrics-client-ca\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.276938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.276893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-textfile\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.276938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.276913    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a1ce953-a8b2-476c-863c-049f522071bf-sys\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.277245 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.277006    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-wtmp\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.277245 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.277027    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a1ce953-a8b2-476c-863c-049f522071bf-root\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.277245 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.277068    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a1ce953-a8b2-476c-863c-049f522071bf-root\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.277245 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.277215    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-textfile\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.277502 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.277468    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a1ce953-a8b2-476c-863c-049f522071bf-metrics-client-ca\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.277502 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.277479    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.280508 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.280484    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-tls\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.280615 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.280595    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a1ce953-a8b2-476c-863c-049f522071bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.289292 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.289262    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:33.327263 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.327232    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdmjs\" (UniqueName: \"kubernetes.io/projected/5a1ce953-a8b2-476c-863c-049f522071bf-kube-api-access-wdmjs\") pod \"node-exporter-d8f74\" (UID: \"5a1ce953-a8b2-476c-863c-049f522071bf\") " pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.393091 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.392985    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d8f74"
Jul 16 21:16:33.403301 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:33.403270    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1ce953_a8b2_476c_863c_049f522071bf.slice/crio-0b7971c868ca4ddeec28e74b64548fa71c2bbd6a5b31ea62cc9d7c8d31bf9e12 WatchSource:0}: Error finding container 0b7971c868ca4ddeec28e74b64548fa71c2bbd6a5b31ea62cc9d7c8d31bf9e12: Status 404 returned error can't find the container with id 0b7971c868ca4ddeec28e74b64548fa71c2bbd6a5b31ea62cc9d7c8d31bf9e12
Jul 16 21:16:33.550733 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.550694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:33.639066 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.639007    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:33.731721 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.731652    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gwgdk"
Jul 16 21:16:33.742947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.742921    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:33.765403 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.765370    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d8f74" event={"ID":"5a1ce953-a8b2-476c-863c-049f522071bf","Type":"ContainerStarted","Data":"0b7971c868ca4ddeec28e74b64548fa71c2bbd6a5b31ea62cc9d7c8d31bf9e12"}
Jul 16 21:16:33.872257 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:33.872221    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:34.003548 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.003465    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:34.105423 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.105395    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:16:34.108792 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.108768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.124742 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.124716    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 16 21:16:34.124912 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.124770    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 16 21:16:34.124912 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.124807    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 16 21:16:34.125736 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.125716    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 16 21:16:34.126141 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.126124    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xj2jt\""
Jul 16 21:16:34.126334 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.126318    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 16 21:16:34.126445 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.126406    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 16 21:16:34.128532 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.128514    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 16 21:16:34.128615 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.128602    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 16 21:16:34.132393 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.132375    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 16 21:16:34.167152 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.167116    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:16:34.184406 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184372    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.184579 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184432    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.184579 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.184579 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184536    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.184735 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184599    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.184735 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184703    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.184838 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184748    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-web-config\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.184838 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184782    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.184921 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.184921 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184902    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.185011 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184934    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-config-out\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.185011 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.184987    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-config-volume\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.185138 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.185013    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2j9v\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-kube-api-access-m2j9v\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.186003 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.185981    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:34.286116 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-config-volume\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286286 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286124    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2j9v\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-kube-api-access-m2j9v\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286286 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286151    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286286 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286174    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286286 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286191    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286286 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286206    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286286 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286286 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286307    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-web-config\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286380    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286403    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.286631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.286429    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-config-out\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.288065 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.287172    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.288065 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.287386    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.288065 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.287892    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.290461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.290393    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.290691 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.290629    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.290691 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.290636    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.290823 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.290752    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-web-config\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.290823 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.290768    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.290823 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.290808    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-config-volume\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.290986 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.290914    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.291040 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.291008    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-config-out\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.291040 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.291014    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.298605 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.298583    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2j9v\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-kube-api-access-m2j9v\") pod \"alertmanager-main-0\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.369967 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.369884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:34.419157 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.419128    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:16:34.593193 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.593099    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:16:34.595557 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:34.595526    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5268693_e432_4c95_8fde_13288a8b6bcb.slice/crio-025868f5b2e5ffeecb339823491d547ce4f1383b5300cf04cd9c01a210f0cb3b WatchSource:0}: Error finding container 025868f5b2e5ffeecb339823491d547ce4f1383b5300cf04cd9c01a210f0cb3b: Status 404 returned error can't find the container with id 025868f5b2e5ffeecb339823491d547ce4f1383b5300cf04cd9c01a210f0cb3b
Jul 16 21:16:34.683340 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.683304    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:34.770500 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.770467    2582 generic.go:356] "Generic (PLEG): container finished" podID="5a1ce953-a8b2-476c-863c-049f522071bf" containerID="e43ff482cc1d4024f3c3856efe6812fc713182665bab7c90232368d4325b046b" exitCode=0
Jul 16 21:16:34.770684 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.770567    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d8f74" event={"ID":"5a1ce953-a8b2-476c-863c-049f522071bf","Type":"ContainerDied","Data":"e43ff482cc1d4024f3c3856efe6812fc713182665bab7c90232368d4325b046b"}
Jul 16 21:16:34.771836 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.771812    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerStarted","Data":"025868f5b2e5ffeecb339823491d547ce4f1383b5300cf04cd9c01a210f0cb3b"}
Jul 16 21:16:34.825103 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.825059    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:34.939480 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:34.939456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:35.701644 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:35.701558    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85fbb945c6-fdbjg"
Jul 16 21:16:35.777956 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:35.777919    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d8f74" event={"ID":"5a1ce953-a8b2-476c-863c-049f522071bf","Type":"ContainerStarted","Data":"b0e02cfe6826f704a705af5c2e3addedcb0f9c78daccf2a646e5d402e5ea3ca1"}
Jul 16 21:16:35.778173 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:35.777986    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d8f74" event={"ID":"5a1ce953-a8b2-476c-863c-049f522071bf","Type":"ContainerStarted","Data":"2148e00caa042e113c0aa57a956fa8a7cfb1ba81a12279e7b398b36e46bee92b"}
Jul 16 21:16:35.870515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:35.870466    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d8f74" podStartSLOduration=2.844074458 podStartE2EDuration="3.870448041s" totalImagesPullingTime="944.721626ms" totalInitContainerRuntime="81.651957ms" isStatefulPod=true podCreationTimestamp="2026-07-16 21:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:35.870119368 +0000 UTC m=+59.926950279" watchObservedRunningTime="2026-07-16 21:16:35.870448041 +0000 UTC m=+59.927279005"
Jul 16 21:16:36.197300 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:36.197262    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:36.305693 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:36.305659    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:36.618490 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:36.618443    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:36.782513 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:36.782482    2582 generic.go:356] "Generic (PLEG): container finished" podID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerID="a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f" exitCode=0
Jul 16 21:16:36.782883 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:36.782565    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerDied","Data":"a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f"}
Jul 16 21:16:36.981251 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:36.981168    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:37.143676 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.143628    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:37.718497 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.718455    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:37.777532 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.777502    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"]
Jul 16 21:16:37.779901 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.779883    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:37.785764 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.785736    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kkdbk\""
Jul 16 21:16:37.796623 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.796604    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 16 21:16:37.807549 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.807521    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"]
Jul 16 21:16:37.826597 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.826569    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4frsm\" (UniqueName: \"kubernetes.io/projected/e3021cbf-7924-46fb-b841-196b3e61b20d-kube-api-access-4frsm\") pod \"monitoring-plugin-7b7b7bc8bf-627gf\" (UID: \"e3021cbf-7924-46fb-b841-196b3e61b20d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:37.826740 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.826724    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3021cbf-7924-46fb-b841-196b3e61b20d-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-627gf\" (UID: \"e3021cbf-7924-46fb-b841-196b3e61b20d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:37.881621 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.881585    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b6c6bb4db-8cb84"]
Jul 16 21:16:37.927747 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.927719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3021cbf-7924-46fb-b841-196b3e61b20d-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-627gf\" (UID: \"e3021cbf-7924-46fb-b841-196b3e61b20d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:37.927923 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.927763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4frsm\" (UniqueName: \"kubernetes.io/projected/e3021cbf-7924-46fb-b841-196b3e61b20d-kube-api-access-4frsm\") pod \"monitoring-plugin-7b7b7bc8bf-627gf\" (UID: \"e3021cbf-7924-46fb-b841-196b3e61b20d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:37.930248 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.930224    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3021cbf-7924-46fb-b841-196b3e61b20d-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-627gf\" (UID: \"e3021cbf-7924-46fb-b841-196b3e61b20d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:37.995800 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:37.995647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:38.002506 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.002479    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4frsm\" (UniqueName: \"kubernetes.io/projected/e3021cbf-7924-46fb-b841-196b3e61b20d-kube-api-access-4frsm\") pod \"monitoring-plugin-7b7b7bc8bf-627gf\" (UID: \"e3021cbf-7924-46fb-b841-196b3e61b20d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:38.089312 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.089272    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:38.098311 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.098279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:38.457396 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.456963    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"]
Jul 16 21:16:38.460350 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:38.460326    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3021cbf_7924_46fb_b841_196b3e61b20d.slice/crio-972d4d1e22ff49f60cd0eedbcaa9173f31d7f124be8beaca68b35a205dc18d6a WatchSource:0}: Error finding container 972d4d1e22ff49f60cd0eedbcaa9173f31d7f124be8beaca68b35a205dc18d6a: Status 404 returned error can't find the container with id 972d4d1e22ff49f60cd0eedbcaa9173f31d7f124be8beaca68b35a205dc18d6a
Jul 16 21:16:38.792487 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.792452    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerStarted","Data":"fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453"}
Jul 16 21:16:38.792942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.792524    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerStarted","Data":"ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1"}
Jul 16 21:16:38.792942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.792567    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerStarted","Data":"0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988"}
Jul 16 21:16:38.792942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.792597    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerStarted","Data":"1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb"}
Jul 16 21:16:38.792942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.792620    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerStarted","Data":"b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e"}
Jul 16 21:16:38.793704 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:38.793681    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf" event={"ID":"e3021cbf-7924-46fb-b841-196b3e61b20d","Type":"ContainerStarted","Data":"972d4d1e22ff49f60cd0eedbcaa9173f31d7f124be8beaca68b35a205dc18d6a"}
Jul 16 21:16:39.675505 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.675477    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xhp"
Jul 16 21:16:39.801757 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.801712    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerStarted","Data":"a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c"}
Jul 16 21:16:39.819157 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.819129    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:39.835470 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.835436    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:16:39.838174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.838158    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.852282 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852251    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 21:16:39.852413 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852309    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 21:16:39.852413 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852355    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 21:16:39.852413 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852251    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 21:16:39.852413 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852405    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 21:16:39.852413 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852371    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 21:16:39.852670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852426    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 21:16:39.852670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852524    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 21:16:39.852670 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852661    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d22n4jl339q5l\""
Jul 16 21:16:39.852812 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852768    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 21:16:39.852894 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852878    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 21:16:39.852951 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852885    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 21:16:39.853001 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.852924    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-r6vng\""
Jul 16 21:16:39.854527 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.854510    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 21:16:39.854761 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.854748    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 21:16:39.857177 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.857159    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 21:16:39.861219 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.861202    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 21:16:39.864774 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.864733    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.589095637 podStartE2EDuration="5.864720728s" totalImagesPullingTime="4.255397958s" totalInitContainerRuntime="20.227133ms" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:39.864337951 +0000 UTC m=+63.921168866" watchObservedRunningTime="2026-07-16 21:16:39.864720728 +0000 UTC m=+63.921551640"
Jul 16 21:16:39.890902 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.890864    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:16:39.945851 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.945769    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.945851 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.945814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.945851 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.945843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-config\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946142 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.945909    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946142 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.945982    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-config-out\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946142 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946007    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946142 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946339 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946149    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946339 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946208    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946339 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946256    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946339 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946282    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946339 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946307    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946347    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946372    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946399    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946430    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-web-config\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946478    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7nz7\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-kube-api-access-t7nz7\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946505    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:39.946578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:39.946530    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.047754 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.047722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.047754 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.047758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.047963 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.047777    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-config\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.047963 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.047795    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.047963 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.047838    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-config-out\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.047963 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.047864    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.047963 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.047886    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.047963 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.047930    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.047973    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048018    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048045    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048126    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048151    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048183    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048206    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048226    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-web-config\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7nz7\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-kube-api-access-t7nz7\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048270    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048934 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048934 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048328    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.048934 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.048516    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.053260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.050333    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.053260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.051655    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.053260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.052140    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.053260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.052555    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-config-out\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.053260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.053207    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.053593 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.053350    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.054066 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.054032    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-web-config\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.054455 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.054400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.057255 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.056744    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.057255 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.057049    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-config\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.057255 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.057147    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.057561 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.057535    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.057627 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.057572    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.057678 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.057653    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.059563 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.058203    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.059563 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.058240    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.059563 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.058573    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.073036 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.073014    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7nz7\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-kube-api-access-t7nz7\") pod \"prometheus-k8s-0\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.147514 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.147478    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:16:40.203015 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.202977    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:40.346704 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.346668    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:16:40.391969 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.391941    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:40.511683 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:40.511611    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a408009_5837_4ceb_8aa8_93b044b85b40.slice/crio-d6f54421bf2a20759766b81339963f4f4a982c740642c74fa28f946013c81272 WatchSource:0}: Error finding container d6f54421bf2a20759766b81339963f4f4a982c740642c74fa28f946013c81272: Status 404 returned error can't find the container with id d6f54421bf2a20759766b81339963f4f4a982c740642c74fa28f946013c81272
Jul 16 21:16:40.807337 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.807304    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerID="ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852" exitCode=0
Jul 16 21:16:40.807756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.807398    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerDied","Data":"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"}
Jul 16 21:16:40.807756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.807487    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerStarted","Data":"d6f54421bf2a20759766b81339963f4f4a982c740642c74fa28f946013c81272"}
Jul 16 21:16:40.809052 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.809025    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf" event={"ID":"e3021cbf-7924-46fb-b841-196b3e61b20d","Type":"ContainerStarted","Data":"8562690712b05783118115ea79ff589743dcae4d0ef615430199a2e7a000baf7"}
Jul 16 21:16:40.809584 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.809562    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:40.814173 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:40.814143    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf"
Jul 16 21:16:42.164848 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.164815    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:42.268983 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.268946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:42.269164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.269044    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:42.272672 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.272642    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 21:16:42.282211 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.282181    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0265e540-9cd6-45da-aa44-bc5d8a5a9a42-metrics-certs\") pod \"network-metrics-daemon-hlv9v\" (UID: \"0265e540-9cd6-45da-aa44-bc5d8a5a9a42\") " pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:42.295951 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.295925    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 21:16:42.301860 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.301839    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/def9cc97-3b3b-455a-a095-21a14521bb12-original-pull-secret\") pod \"global-pull-secret-syncer-h98vh\" (UID: \"def9cc97-3b3b-455a-a095-21a14521bb12\") " pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:42.319283 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.319248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:42.370557 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.370522    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:42.373482 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.373462    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 21:16:42.385462 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.385441    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 21:16:42.395067 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.395041    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msnzf\" (UniqueName: \"kubernetes.io/projected/1b5ce203-55c6-4d32-92d1-a39eb10a0d2f-kube-api-access-msnzf\") pod \"network-check-target-kqszq\" (UID: \"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f\") " pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:42.523220 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.523143    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h98vh"
Jul 16 21:16:42.536230 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.536199    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tpjps\""
Jul 16 21:16:42.542430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.542409    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hlv9v"
Jul 16 21:16:42.553864 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.553839    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzz4k\""
Jul 16 21:16:42.560984 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.560909    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:42.755522 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.755454    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-627gf" podStartSLOduration=3.676881808 podStartE2EDuration="5.755431351s" totalImagesPullingTime="2.078549543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:40.999466733 +0000 UTC m=+65.056297656" watchObservedRunningTime="2026-07-16 21:16:42.755431351 +0000 UTC m=+66.812262262"
Jul 16 21:16:42.757349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.757325    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h98vh"]
Jul 16 21:16:42.761530 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:42.761500    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef9cc97_3b3b_455a_a095_21a14521bb12.slice/crio-d427bc0307f4153fee25822aaf03f33684d96a8feac6ddf2b221343b6c1f33e1 WatchSource:0}: Error finding container d427bc0307f4153fee25822aaf03f33684d96a8feac6ddf2b221343b6c1f33e1: Status 404 returned error can't find the container with id d427bc0307f4153fee25822aaf03f33684d96a8feac6ddf2b221343b6c1f33e1
Jul 16 21:16:42.784472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.784449    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hlv9v"]
Jul 16 21:16:42.786057 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:42.786029    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0265e540_9cd6_45da_aa44_bc5d8a5a9a42.slice/crio-8ab6b06c5053d5ab174f7175e2bc36be6288b7c269823f921e38b7692af0fcab WatchSource:0}: Error finding container 8ab6b06c5053d5ab174f7175e2bc36be6288b7c269823f921e38b7692af0fcab: Status 404 returned error can't find the container with id 8ab6b06c5053d5ab174f7175e2bc36be6288b7c269823f921e38b7692af0fcab
Jul 16 21:16:42.799570 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.799539    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kqszq"]
Jul 16 21:16:42.804283 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:16:42.804251    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5ce203_55c6_4d32_92d1_a39eb10a0d2f.slice/crio-ea46ffe877a780d61d2d63c7589240dc6f4b87cf4480d98a1cf1c6f26dc14300 WatchSource:0}: Error finding container ea46ffe877a780d61d2d63c7589240dc6f4b87cf4480d98a1cf1c6f26dc14300: Status 404 returned error can't find the container with id ea46ffe877a780d61d2d63c7589240dc6f4b87cf4480d98a1cf1c6f26dc14300
Jul 16 21:16:42.818571 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.818533    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h98vh" event={"ID":"def9cc97-3b3b-455a-a095-21a14521bb12","Type":"ContainerStarted","Data":"d427bc0307f4153fee25822aaf03f33684d96a8feac6ddf2b221343b6c1f33e1"}
Jul 16 21:16:42.820097 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.820051    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hlv9v" event={"ID":"0265e540-9cd6-45da-aa44-bc5d8a5a9a42","Type":"ContainerStarted","Data":"8ab6b06c5053d5ab174f7175e2bc36be6288b7c269823f921e38b7692af0fcab"}
Jul 16 21:16:42.821377 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:42.821347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kqszq" event={"ID":"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f","Type":"ContainerStarted","Data":"ea46ffe877a780d61d2d63c7589240dc6f4b87cf4480d98a1cf1c6f26dc14300"}
Jul 16 21:16:44.002427 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:44.002389    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:44.142007 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:44.141921    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:44.317390 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:44.317341    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:47.244456 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:47.244416    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:47.359717 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:47.359659    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:48.849709 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:48.849672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hlv9v" event={"ID":"0265e540-9cd6-45da-aa44-bc5d8a5a9a42","Type":"ContainerStarted","Data":"1501ce8a8499cbd00038eb84ebb427f2ecfb48cbb97cd955ac039a62437295ed"}
Jul 16 21:16:48.850233 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:48.849738    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hlv9v" event={"ID":"0265e540-9cd6-45da-aa44-bc5d8a5a9a42","Type":"ContainerStarted","Data":"adcdc315670fb6a59fae99afac2d9bc7acb428ec8ec29acfc537a051ab096035"}
Jul 16 21:16:48.853422 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:48.853379    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kqszq" event={"ID":"1b5ce203-55c6-4d32-92d1-a39eb10a0d2f","Type":"ContainerStarted","Data":"e38f2916d09a5d50b92e4bf73d84a05696ebe0b2232119f53147f721bbedc5aa"}
Jul 16 21:16:48.854119 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:48.854063    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:16:48.857456 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:48.857421    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h98vh" event={"ID":"def9cc97-3b3b-455a-a095-21a14521bb12","Type":"ContainerStarted","Data":"b247f0f6f2dc100d9e6ad9d65e28b8ed954669bbcaba33d070218c80c4842a1d"}
Jul 16 21:16:48.859755 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:48.859733    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerStarted","Data":"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"}
Jul 16 21:16:48.859863 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:48.859783    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerStarted","Data":"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"}
Jul 16 21:16:48.889459 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:48.889405    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hlv9v" podStartSLOduration=66.812731979 podStartE2EDuration="1m11.889375641s" totalImagesPullingTime="5.076643662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:48.887596753 +0000 UTC m=+72.944427665" watchObservedRunningTime="2026-07-16 21:16:48.889375641 +0000 UTC m=+72.946206558"
Jul 16 21:16:48.968719 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:48.968654    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kqszq" podStartSLOduration=66.879212703 podStartE2EDuration="1m11.968632228s" totalImagesPullingTime="5.089419525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:48.932239045 +0000 UTC m=+72.989069957" watchObservedRunningTime="2026-07-16 21:16:48.968632228 +0000 UTC m=+73.025463139"
Jul 16 21:16:49.080595 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:49.080563    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:49.182252 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:49.182172    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:49.824596 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:49.824562    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:49.963351 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:49.963317    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:50.873048 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:50.873017    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerStarted","Data":"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"}
Jul 16 21:16:50.873251 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:50.873123    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerStarted","Data":"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"}
Jul 16 21:16:50.873251 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:50.873167    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerStarted","Data":"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"}
Jul 16 21:16:50.873251 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:50.873203    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerStarted","Data":"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"}
Jul 16 21:16:50.927767 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:50.927709    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h98vh" podStartSLOduration=68.782468198 podStartE2EDuration="1m13.92768997s" totalImagesPullingTime="5.145221772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:48.969877846 +0000 UTC m=+73.026708757" watchObservedRunningTime="2026-07-16 21:16:50.92768997 +0000 UTC m=+74.984520901"
Jul 16 21:16:50.928163 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:50.928130    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.704559277 podStartE2EDuration="11.928115267s" totalImagesPullingTime="9.203300218s" totalInitContainerRuntime="20.255772ms" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:50.926621287 +0000 UTC m=+74.983452200" watchObservedRunningTime="2026-07-16 21:16:50.928115267 +0000 UTC m=+74.984946180"
Jul 16 21:16:51.075738 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:51.075707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:51.168461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:51.168379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:16:55.148036 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:16:55.148001    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:02.905205 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:02.905052    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b6c6bb4db-8cb84" podUID="69590fa3-f855-4cb8-b04e-e791bafe13ec" containerName="console" containerID="cri-o://4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c" gracePeriod=15
Jul 16 21:17:03.196400 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.196363    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:17:03.364190 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364162    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-service-ca\") pod \"69590fa3-f855-4cb8-b04e-e791bafe13ec\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") "
Jul 16 21:17:03.364369 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364215    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-oauth-serving-cert\") pod \"69590fa3-f855-4cb8-b04e-e791bafe13ec\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") "
Jul 16 21:17:03.364369 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364237    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-serving-cert\") pod \"69590fa3-f855-4cb8-b04e-e791bafe13ec\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") "
Jul 16 21:17:03.364369 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364353    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwpkk\" (UniqueName: \"kubernetes.io/projected/69590fa3-f855-4cb8-b04e-e791bafe13ec-kube-api-access-pwpkk\") pod \"69590fa3-f855-4cb8-b04e-e791bafe13ec\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") "
Jul 16 21:17:03.364541 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364399    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-oauth-config\") pod \"69590fa3-f855-4cb8-b04e-e791bafe13ec\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") "
Jul 16 21:17:03.364541 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364452    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-config\") pod \"69590fa3-f855-4cb8-b04e-e791bafe13ec\" (UID: \"69590fa3-f855-4cb8-b04e-e791bafe13ec\") "
Jul 16 21:17:03.364636 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364568    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-service-ca" (OuterVolumeSpecName: "service-ca") pod "69590fa3-f855-4cb8-b04e-e791bafe13ec" (UID: "69590fa3-f855-4cb8-b04e-e791bafe13ec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:03.364689 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364659    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "69590fa3-f855-4cb8-b04e-e791bafe13ec" (UID: "69590fa3-f855-4cb8-b04e-e791bafe13ec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:03.364847 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364806    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:03.364847 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364822    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-config" (OuterVolumeSpecName: "console-config") pod "69590fa3-f855-4cb8-b04e-e791bafe13ec" (UID: "69590fa3-f855-4cb8-b04e-e791bafe13ec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:03.364847 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.364828    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-oauth-serving-cert\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:03.366829 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.366795    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "69590fa3-f855-4cb8-b04e-e791bafe13ec" (UID: "69590fa3-f855-4cb8-b04e-e791bafe13ec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:03.366829 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.366813    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69590fa3-f855-4cb8-b04e-e791bafe13ec-kube-api-access-pwpkk" (OuterVolumeSpecName: "kube-api-access-pwpkk") pod "69590fa3-f855-4cb8-b04e-e791bafe13ec" (UID: "69590fa3-f855-4cb8-b04e-e791bafe13ec"). InnerVolumeSpecName "kube-api-access-pwpkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:17:03.367198 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.367173    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "69590fa3-f855-4cb8-b04e-e791bafe13ec" (UID: "69590fa3-f855-4cb8-b04e-e791bafe13ec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:03.465472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.465395    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwpkk\" (UniqueName: \"kubernetes.io/projected/69590fa3-f855-4cb8-b04e-e791bafe13ec-kube-api-access-pwpkk\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:03.465472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.465424    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-oauth-config\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:03.465472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.465434    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-config\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:03.465472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.465442    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69590fa3-f855-4cb8-b04e-e791bafe13ec-console-serving-cert\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:03.928128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.924841    2582 generic.go:356] "Generic (PLEG): container finished" podID="69590fa3-f855-4cb8-b04e-e791bafe13ec" containerID="4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c" exitCode=0
Jul 16 21:17:03.928128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.924926    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6c6bb4db-8cb84" event={"ID":"69590fa3-f855-4cb8-b04e-e791bafe13ec","Type":"ContainerDied","Data":"4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c"}
Jul 16 21:17:03.928128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.924981    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6c6bb4db-8cb84" event={"ID":"69590fa3-f855-4cb8-b04e-e791bafe13ec","Type":"ContainerDied","Data":"0af22b1195b474b48834e7bd4eb210ba02553a97538908158fed983daf6cc2a5"}
Jul 16 21:17:03.928128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.925009    2582 scope.go:117] "RemoveContainer" containerID="4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c"
Jul 16 21:17:03.928128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.925220    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6c6bb4db-8cb84"
Jul 16 21:17:03.941858 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.941830    2582 scope.go:117] "RemoveContainer" containerID="4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c"
Jul 16 21:17:03.942219 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:03.942191    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c\": container with ID starting with 4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c not found: ID does not exist" containerID="4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c"
Jul 16 21:17:03.942341 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.942231    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c"} err="failed to get container status \"4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c\": rpc error: code = NotFound desc = could not find container \"4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c\": container with ID starting with 4b64d631bbf273cc4813020df980eb11b8c2becb0a1df879950dca02b80dcf5c not found: ID does not exist"
Jul 16 21:17:03.971884 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.971848    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b6c6bb4db-8cb84"]
Jul 16 21:17:03.980431 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:03.980403    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b6c6bb4db-8cb84"]
Jul 16 21:17:04.506113 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:04.506061    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69590fa3-f855-4cb8-b04e-e791bafe13ec" path="/var/lib/kubelet/pods/69590fa3-f855-4cb8-b04e-e791bafe13ec/volumes"
Jul 16 21:17:19.822716 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:19.822683    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:17:19.892061 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:19.892029    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:17:20.875505 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:20.875471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kqszq"
Jul 16 21:17:22.086449 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:22.086416    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:17:35.934461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:35.934389    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:17:40.148456 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:40.148421    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:40.168609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:40.168584    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:41.079845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:41.079819    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:52.675948 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:52.675889    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:17:52.820128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:52.820092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:17:53.652143 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:53.652106    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:17:53.652433 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:53.652412    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="alertmanager" containerID="cri-o://b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e" gracePeriod=120
Jul 16 21:17:53.652534 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:53.652474    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy-metric" containerID="cri-o://fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453" gracePeriod=120
Jul 16 21:17:53.652763 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:53.652510    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="prom-label-proxy" containerID="cri-o://a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c" gracePeriod=120
Jul 16 21:17:53.652846 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:53.652541    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy-web" containerID="cri-o://0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988" gracePeriod=120
Jul 16 21:17:53.653331 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:53.652574    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy" containerID="cri-o://ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1" gracePeriod=120
Jul 16 21:17:53.653331 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:53.652574    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="config-reloader" containerID="cri-o://1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb" gracePeriod=120
Jul 16 21:17:54.112309 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112278    2582 generic.go:356] "Generic (PLEG): container finished" podID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerID="a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c" exitCode=0
Jul 16 21:17:54.112309 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112302    2582 generic.go:356] "Generic (PLEG): container finished" podID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerID="fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453" exitCode=0
Jul 16 21:17:54.112309 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112308    2582 generic.go:356] "Generic (PLEG): container finished" podID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerID="ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1" exitCode=0
Jul 16 21:17:54.112309 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112316    2582 generic.go:356] "Generic (PLEG): container finished" podID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerID="1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb" exitCode=0
Jul 16 21:17:54.112765 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112327    2582 generic.go:356] "Generic (PLEG): container finished" podID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerID="b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e" exitCode=0
Jul 16 21:17:54.112765 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112345    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerDied","Data":"a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c"}
Jul 16 21:17:54.112765 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112404    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerDied","Data":"fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453"}
Jul 16 21:17:54.112765 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112430    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerDied","Data":"ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1"}
Jul 16 21:17:54.112765 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112454    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerDied","Data":"1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb"}
Jul 16 21:17:54.112765 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.112477    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerDied","Data":"b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e"}
Jul 16 21:17:54.903181 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.903147    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:54.969051 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969024    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-main-db\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969065    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-web\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969126    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-trusted-ca-bundle\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969152    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969179    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-web-config\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969214    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969274    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-tls-assets\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969296    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2j9v\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-kube-api-access-m2j9v\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969322    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-metrics-client-ca\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969359    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-cluster-tls-config\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969386    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-config-out\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969420    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-main-tls\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969450    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-config-volume\") pod \"a5268693-e432-4c95-8fde-13288a8b6bcb\" (UID: \"a5268693-e432-4c95-8fde-13288a8b6bcb\") "
Jul 16 21:17:54.969507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969455    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:17:54.969890 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969690    2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-main-db\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:54.969890 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969685    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:54.969890 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.969761    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:54.973540 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.973505    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:54.974423 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.974391    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-config-volume" (OuterVolumeSpecName: "config-volume") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:54.974555 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.974439    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-kube-api-access-m2j9v" (OuterVolumeSpecName: "kube-api-access-m2j9v") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "kube-api-access-m2j9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:17:54.974555 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.974496    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-config-out" (OuterVolumeSpecName: "config-out") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:17:54.974668 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.974629    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:54.974746 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.974726    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:54.975094 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.975051    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:17:54.975906 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.975881    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:54.977432 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.977411    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:54.984877 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:54.984850    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-web-config" (OuterVolumeSpecName: "web-config") pod "a5268693-e432-4c95-8fde-13288a8b6bcb" (UID: "a5268693-e432-4c95-8fde-13288a8b6bcb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:55.070403 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070366    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-main-tls\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070403 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070402    2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-config-volume\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070413    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070425    2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070435    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070466    2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-web-config\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070475    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070484    2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-tls-assets\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070492    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2j9v\" (UniqueName: \"kubernetes.io/projected/a5268693-e432-4c95-8fde-13288a8b6bcb-kube-api-access-m2j9v\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070502    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5268693-e432-4c95-8fde-13288a8b6bcb-metrics-client-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070511    2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a5268693-e432-4c95-8fde-13288a8b6bcb-cluster-tls-config\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.070601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.070518    2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5268693-e432-4c95-8fde-13288a8b6bcb-config-out\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:55.119655 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.119625    2582 generic.go:356] "Generic (PLEG): container finished" podID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerID="0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988" exitCode=0
Jul 16 21:17:55.120023 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.119681    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerDied","Data":"0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988"}
Jul 16 21:17:55.120023 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.119696    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.120023 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.119742    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5268693-e432-4c95-8fde-13288a8b6bcb","Type":"ContainerDied","Data":"025868f5b2e5ffeecb339823491d547ce4f1383b5300cf04cd9c01a210f0cb3b"}
Jul 16 21:17:55.120023 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.119774    2582 scope.go:117] "RemoveContainer" containerID="a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c"
Jul 16 21:17:55.128199 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.128052    2582 scope.go:117] "RemoveContainer" containerID="fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453"
Jul 16 21:17:55.138062 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.137881    2582 scope.go:117] "RemoveContainer" containerID="ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1"
Jul 16 21:17:55.145072 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.145052    2582 scope.go:117] "RemoveContainer" containerID="0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988"
Jul 16 21:17:55.150503 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.150479    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:17:55.153935 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.153726    2582 scope.go:117] "RemoveContainer" containerID="1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb"
Jul 16 21:17:55.155909 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.155889    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:17:55.161447 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.161418    2582 scope.go:117] "RemoveContainer" containerID="b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e"
Jul 16 21:17:55.169061 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.169043    2582 scope.go:117] "RemoveContainer" containerID="a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f"
Jul 16 21:17:55.175818 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.175795    2582 scope.go:117] "RemoveContainer" containerID="a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c"
Jul 16 21:17:55.176103 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:55.176064    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c\": container with ID starting with a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c not found: ID does not exist" containerID="a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c"
Jul 16 21:17:55.176157 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.176114    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c"} err="failed to get container status \"a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c\": rpc error: code = NotFound desc = could not find container \"a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c\": container with ID starting with a784abfc315a6177156a615210f8918d492002d577201f4fa31f8ac1dac6f18c not found: ID does not exist"
Jul 16 21:17:55.176157 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.176133    2582 scope.go:117] "RemoveContainer" containerID="fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453"
Jul 16 21:17:55.176350 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:55.176332    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453\": container with ID starting with fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453 not found: ID does not exist" containerID="fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453"
Jul 16 21:17:55.176457 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.176351    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453"} err="failed to get container status \"fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453\": rpc error: code = NotFound desc = could not find container \"fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453\": container with ID starting with fadca17f8c9ea8fb9c8546db71018adbcca88dc339356cbf8ae3a8cd05182453 not found: ID does not exist"
Jul 16 21:17:55.176457 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.176362    2582 scope.go:117] "RemoveContainer" containerID="ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1"
Jul 16 21:17:55.176595 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:55.176581    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1\": container with ID starting with ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1 not found: ID does not exist" containerID="ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1"
Jul 16 21:17:55.176637 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.176598    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1"} err="failed to get container status \"ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1\": rpc error: code = NotFound desc = could not find container \"ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1\": container with ID starting with ca562407174752b6c8a6530074e6d1a9a0026e54c00951b73d5f17c1f86834e1 not found: ID does not exist"
Jul 16 21:17:55.176637 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.176610    2582 scope.go:117] "RemoveContainer" containerID="0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988"
Jul 16 21:17:55.176842 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:55.176824    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988\": container with ID starting with 0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988 not found: ID does not exist" containerID="0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988"
Jul 16 21:17:55.176892 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.176847    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988"} err="failed to get container status \"0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988\": rpc error: code = NotFound desc = could not find container \"0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988\": container with ID starting with 0817ad4669c273e07058cdd4c92f044de798cdb111021e8cef0cfac632fb4988 not found: ID does not exist"
Jul 16 21:17:55.176892 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.176864    2582 scope.go:117] "RemoveContainer" containerID="1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb"
Jul 16 21:17:55.177091 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:55.177063    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb\": container with ID starting with 1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb not found: ID does not exist" containerID="1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb"
Jul 16 21:17:55.177141 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.177095    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb"} err="failed to get container status \"1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb\": rpc error: code = NotFound desc = could not find container \"1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb\": container with ID starting with 1dd77fb8606179a8c052f22b415383fae2745d4fa49648db727af92519345abb not found: ID does not exist"
Jul 16 21:17:55.177141 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.177108    2582 scope.go:117] "RemoveContainer" containerID="b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e"
Jul 16 21:17:55.177334 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:55.177317    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e\": container with ID starting with b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e not found: ID does not exist" containerID="b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e"
Jul 16 21:17:55.177378 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.177336    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e"} err="failed to get container status \"b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e\": rpc error: code = NotFound desc = could not find container \"b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e\": container with ID starting with b5db6a054fa4f98eea19dd4e7a8c7de8d4396bc28976cf56b7195dc63df48c3e not found: ID does not exist"
Jul 16 21:17:55.177378 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.177347    2582 scope.go:117] "RemoveContainer" containerID="a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f"
Jul 16 21:17:55.177557 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:55.177543    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f\": container with ID starting with a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f not found: ID does not exist" containerID="a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f"
Jul 16 21:17:55.177592 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.177559    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f"} err="failed to get container status \"a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f\": rpc error: code = NotFound desc = could not find container \"a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f\": container with ID starting with a254aa94270cb4ba93cff4f255af15b0a7bbea43a8a925002206d49263b7bc9f not found: ID does not exist"
Jul 16 21:17:55.197391 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197364    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:17:55.197642 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197630    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy-metric"
Jul 16 21:17:55.197691 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197644    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy-metric"
Jul 16 21:17:55.197691 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197657    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69590fa3-f855-4cb8-b04e-e791bafe13ec" containerName="console"
Jul 16 21:17:55.197691 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197662    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="69590fa3-f855-4cb8-b04e-e791bafe13ec" containerName="console"
Jul 16 21:17:55.197691 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197669    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy"
Jul 16 21:17:55.197691 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197675    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy"
Jul 16 21:17:55.197691 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197683    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="config-reloader"
Jul 16 21:17:55.197691 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197688    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="config-reloader"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197695    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="prom-label-proxy"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197700    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="prom-label-proxy"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197709    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy-web"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197714    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy-web"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197722    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="alertmanager"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197727    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="alertmanager"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197732    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="init-config-reloader"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197737    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="init-config-reloader"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197774    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="prom-label-proxy"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197780    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="alertmanager"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197786    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197792    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="69590fa3-f855-4cb8-b04e-e791bafe13ec" containerName="console"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197798    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="config-reloader"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197805    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy-web"
Jul 16 21:17:55.197887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.197810    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" containerName="kube-rbac-proxy-metric"
Jul 16 21:17:55.201465 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.201451    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.206070 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.206050    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 16 21:17:55.206325 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.206310    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xj2jt\""
Jul 16 21:17:55.206858 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.206720    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 16 21:17:55.206858 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.206743    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 16 21:17:55.206858 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.206764    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 16 21:17:55.206858 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.206776    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 16 21:17:55.206858 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.206783    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 16 21:17:55.207199 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.207108    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 16 21:17:55.207199 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.207143    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 16 21:17:55.215353 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.215332    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 16 21:17:55.222497 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.222474    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:17:55.271979 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.271943    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8zbr\" (UniqueName: \"kubernetes.io/projected/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-kube-api-access-j8zbr\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.271979 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.271981    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-web-config\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272230 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.271999    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272230 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272097    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272230 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272136    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272230 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272160    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272230 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272400 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272246    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272400 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272400 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272305    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272400 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272325    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272400 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272339    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-config-out\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.272400 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.272371    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373581 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373502    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8zbr\" (UniqueName: \"kubernetes.io/projected/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-kube-api-access-j8zbr\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373581 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-web-config\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373581 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373785 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373702    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373785 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373785 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373780    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373863    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373891    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.373938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373921    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.374194 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373953    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.374194 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.373981    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-config-out\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.374194 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.374036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.374347 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.374248    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.374347 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.374331    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.374938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.374859    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.377221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.377026    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.377221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.377026    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-web-config\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.377221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.377167    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.377221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.377201    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.377491 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.377279    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.377491 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.377315    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.377491 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.377417    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.377491 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.377434    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.378850 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.378833    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-config-out\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.384023 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.384000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8zbr\" (UniqueName: \"kubernetes.io/projected/3ec5e9f1-0527-4d99-bcdf-50b19b71f633-kube-api-access-j8zbr\") pod \"alertmanager-main-0\" (UID: \"3ec5e9f1-0527-4d99-bcdf-50b19b71f633\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.510711 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.510674    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:17:55.670126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:55.670045    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:17:55.672291 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:17:55.672266    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec5e9f1_0527_4d99_bcdf_50b19b71f633.slice/crio-bc28f9c774afb1db6b51ab87ac4fc220945d027afe4413e365fc4be9dd7cb6e1 WatchSource:0}: Error finding container bc28f9c774afb1db6b51ab87ac4fc220945d027afe4413e365fc4be9dd7cb6e1: Status 404 returned error can't find the container with id bc28f9c774afb1db6b51ab87ac4fc220945d027afe4413e365fc4be9dd7cb6e1
Jul 16 21:17:56.125036 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:56.125004    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ec5e9f1-0527-4d99-bcdf-50b19b71f633" containerID="30ebe7ece19e2dd40c93d3374d0d575eca950f8180ba15e972903ad44b1b8d6f" exitCode=0
Jul 16 21:17:56.125499 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:56.125110    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ec5e9f1-0527-4d99-bcdf-50b19b71f633","Type":"ContainerDied","Data":"30ebe7ece19e2dd40c93d3374d0d575eca950f8180ba15e972903ad44b1b8d6f"}
Jul 16 21:17:56.125499 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:56.125189    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ec5e9f1-0527-4d99-bcdf-50b19b71f633","Type":"ContainerStarted","Data":"bc28f9c774afb1db6b51ab87ac4fc220945d027afe4413e365fc4be9dd7cb6e1"}
Jul 16 21:17:56.505237 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:56.505201    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5268693-e432-4c95-8fde-13288a8b6bcb" path="/var/lib/kubelet/pods/a5268693-e432-4c95-8fde-13288a8b6bcb/volumes"
Jul 16 21:17:57.132832 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:57.132795    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ec5e9f1-0527-4d99-bcdf-50b19b71f633","Type":"ContainerStarted","Data":"fe9cb0e10e9dd06217b83e538b2e3490fda2fe7b340be1366f41776aa1dfff7d"}
Jul 16 21:17:57.133259 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:57.132866    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ec5e9f1-0527-4d99-bcdf-50b19b71f633","Type":"ContainerStarted","Data":"96caa98edd338bbc7ed31a5d2f87b0ac682c5801e11323c04be40a3c763d69ff"}
Jul 16 21:17:57.133259 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:57.132905    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ec5e9f1-0527-4d99-bcdf-50b19b71f633","Type":"ContainerStarted","Data":"46adec1540a3f1763791ca72bb49b918ee8d3424a90c2313ff6bef7d6de52c17"}
Jul 16 21:17:57.133259 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:57.132933    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ec5e9f1-0527-4d99-bcdf-50b19b71f633","Type":"ContainerStarted","Data":"7373c92a7a8b4ca9cd269ec94eebad1ccbf6cd717231d9d1f99e0afef79f15b8"}
Jul 16 21:17:57.133259 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:57.132969    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ec5e9f1-0527-4d99-bcdf-50b19b71f633","Type":"ContainerStarted","Data":"6963b4585f2685553b8391d05f6317cb1c2ba66d4f48bc2d6443d44a80eb0316"}
Jul 16 21:17:57.133259 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:57.133015    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ec5e9f1-0527-4d99-bcdf-50b19b71f633","Type":"ContainerStarted","Data":"01eb46d9966c71967b3dc704e9e7054ff310f929055440b67a31695a8d519b62"}
Jul 16 21:17:57.168564 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:57.168510    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.152924184 podStartE2EDuration="2.168487831s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.563647ms" isStatefulPod=false podCreationTimestamp="2026-07-16 21:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:17:57.167345803 +0000 UTC m=+141.224176710" watchObservedRunningTime="2026-07-16 21:17:57.168487831 +0000 UTC m=+141.225318744"
Jul 16 21:17:58.160969 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.160930    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:17:58.161485 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.161331    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="prometheus" containerID="cri-o://5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4" gracePeriod=600
Jul 16 21:17:58.161485 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.161348    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="thanos-sidecar" containerID="cri-o://d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c" gracePeriod=600
Jul 16 21:17:58.161485 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.161373    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0" gracePeriod=600
Jul 16 21:17:58.161485 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.161374    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy-web" containerID="cri-o://dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340" gracePeriod=600
Jul 16 21:17:58.161727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.161386    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="config-reloader" containerID="cri-o://033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c" gracePeriod=600
Jul 16 21:17:58.161727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.161346    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy" containerID="cri-o://7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881" gracePeriod=600
Jul 16 21:17:58.411114 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.411032    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:58.499696 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499661    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-db\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.499869 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499715    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-thanos-prometheus-http-client-file\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.499869 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499768    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.499869 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499795    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-serving-certs-ca-bundle\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.499869 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499836    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-metrics-client-certs\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.499869 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499864    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-tls\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499891    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-2\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499917    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-trusted-ca-bundle\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499945    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-1\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.499977    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-kube-rbac-proxy\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500010    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-grpc-tls\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500036    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7nz7\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-kube-api-access-t7nz7\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-0\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500118    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-metrics-client-ca\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500160    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-config-out\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500196    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-config\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500246    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-kubelet-serving-ca-bundle\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500271    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-web-config\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500301    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-tls-assets\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500338    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7a408009-5837-4ceb-8aa8-93b044b85b40\" (UID: \"7a408009-5837-4ceb-8aa8-93b044b85b40\") "
Jul 16 21:17:58.500609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.500543    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:58.501502 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.501066    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:58.501502 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.501413    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:17:58.502505 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.502251    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:58.502505 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.502396    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:58.502662 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.502571    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:58.502728 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.502670    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:58.503049 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.502806    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:58.503049 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.502810    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:58.503049 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.502956    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:58.503325 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.503108    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:58.505238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.505163    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-config" (OuterVolumeSpecName: "config") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:58.505843 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.505814    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:58.505843 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.505828    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-kube-api-access-t7nz7" (OuterVolumeSpecName: "kube-api-access-t7nz7") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "kube-api-access-t7nz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:17:58.505989 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.505923    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:17:58.505989 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.505933    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:58.506126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.506103    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:58.507138 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.507055    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:58.507350 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.507330    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-config-out" (OuterVolumeSpecName: "config-out") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:17:58.518798 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.518771    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-web-config" (OuterVolumeSpecName: "web-config") pod "7a408009-5837-4ceb-8aa8-93b044b85b40" (UID: "7a408009-5837-4ceb-8aa8-93b044b85b40"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:58.601349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601310    2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-config\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601342    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601373    2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-web-config\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601382    2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-tls-assets\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601391    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601403    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-db\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601412    2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601421    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601430    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601440    2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-metrics-client-certs\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601450    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601458    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601466    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601475    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601483    2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-kube-rbac-proxy\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601490    2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a408009-5837-4ceb-8aa8-93b044b85b40-secret-grpc-tls\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601498    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7nz7\" (UniqueName: \"kubernetes.io/projected/7a408009-5837-4ceb-8aa8-93b044b85b40-kube-api-access-t7nz7\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601505    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601513    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a408009-5837-4ceb-8aa8-93b044b85b40-configmap-metrics-client-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:58.601980 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:58.601520    2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a408009-5837-4ceb-8aa8-93b044b85b40-config-out\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:17:59.142472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142435    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerID="9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0" exitCode=0
Jul 16 21:17:59.142472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142464    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerID="7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881" exitCode=0
Jul 16 21:17:59.142472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142472    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerID="dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340" exitCode=0
Jul 16 21:17:59.142472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142478    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerID="d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c" exitCode=0
Jul 16 21:17:59.142472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142483    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerID="033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c" exitCode=0
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142488    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerID="5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4" exitCode=0
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142511    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerDied","Data":"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"}
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142528    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142578    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerDied","Data":"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"}
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142608    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerDied","Data":"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"}
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142633    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerDied","Data":"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"}
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142658    2582 scope.go:117] "RemoveContainer" containerID="9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142673    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerDied","Data":"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"}
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142709    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerDied","Data":"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"}
Jul 16 21:17:59.142756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.142747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a408009-5837-4ceb-8aa8-93b044b85b40","Type":"ContainerDied","Data":"d6f54421bf2a20759766b81339963f4f4a982c740642c74fa28f946013c81272"}
Jul 16 21:17:59.150526 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.150476    2582 scope.go:117] "RemoveContainer" containerID="7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"
Jul 16 21:17:59.157625 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.157609    2582 scope.go:117] "RemoveContainer" containerID="dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"
Jul 16 21:17:59.164349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.164209    2582 scope.go:117] "RemoveContainer" containerID="d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"
Jul 16 21:17:59.167352 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.167278    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:17:59.171846 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.171811    2582 scope.go:117] "RemoveContainer" containerID="033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"
Jul 16 21:17:59.173129 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.173109    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:17:59.186319 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.186303    2582 scope.go:117] "RemoveContainer" containerID="5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"
Jul 16 21:17:59.193159 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.193142    2582 scope.go:117] "RemoveContainer" containerID="ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"
Jul 16 21:17:59.199675 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.199659    2582 scope.go:117] "RemoveContainer" containerID="9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"
Jul 16 21:17:59.199944 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:59.199924    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": container with ID starting with 9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0 not found: ID does not exist" containerID="9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"
Jul 16 21:17:59.200016 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.199956    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"} err="failed to get container status \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": rpc error: code = NotFound desc = could not find container \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": container with ID starting with 9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0 not found: ID does not exist"
Jul 16 21:17:59.200016 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.199983    2582 scope.go:117] "RemoveContainer" containerID="7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"
Jul 16 21:17:59.200258 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:59.200242    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": container with ID starting with 7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881 not found: ID does not exist" containerID="7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"
Jul 16 21:17:59.200319 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.200266    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"} err="failed to get container status \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": rpc error: code = NotFound desc = could not find container \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": container with ID starting with 7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881 not found: ID does not exist"
Jul 16 21:17:59.200319 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.200289    2582 scope.go:117] "RemoveContainer" containerID="dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"
Jul 16 21:17:59.200547 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:59.200529    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": container with ID starting with dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340 not found: ID does not exist" containerID="dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"
Jul 16 21:17:59.200613 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.200554    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"} err="failed to get container status \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": rpc error: code = NotFound desc = could not find container \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": container with ID starting with dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340 not found: ID does not exist"
Jul 16 21:17:59.200613 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.200575    2582 scope.go:117] "RemoveContainer" containerID="d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"
Jul 16 21:17:59.200796 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:59.200776    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": container with ID starting with d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c not found: ID does not exist" containerID="d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"
Jul 16 21:17:59.200842 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.200802    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"} err="failed to get container status \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": rpc error: code = NotFound desc = could not find container \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": container with ID starting with d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c not found: ID does not exist"
Jul 16 21:17:59.200842 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.200823    2582 scope.go:117] "RemoveContainer" containerID="033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"
Jul 16 21:17:59.201024 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:59.201006    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": container with ID starting with 033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c not found: ID does not exist" containerID="033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"
Jul 16 21:17:59.201062 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.201030    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"} err="failed to get container status \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": rpc error: code = NotFound desc = could not find container \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": container with ID starting with 033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c not found: ID does not exist"
Jul 16 21:17:59.201062 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.201044    2582 scope.go:117] "RemoveContainer" containerID="5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"
Jul 16 21:17:59.201300 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:59.201284    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": container with ID starting with 5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4 not found: ID does not exist" containerID="5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"
Jul 16 21:17:59.201346 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.201301    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"} err="failed to get container status \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": rpc error: code = NotFound desc = could not find container \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": container with ID starting with 5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4 not found: ID does not exist"
Jul 16 21:17:59.201346 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.201321    2582 scope.go:117] "RemoveContainer" containerID="ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"
Jul 16 21:17:59.201568 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:17:59.201552    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": container with ID starting with ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852 not found: ID does not exist" containerID="ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"
Jul 16 21:17:59.201628 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.201575    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"} err="failed to get container status \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": rpc error: code = NotFound desc = could not find container \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": container with ID starting with ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852 not found: ID does not exist"
Jul 16 21:17:59.201628 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.201594    2582 scope.go:117] "RemoveContainer" containerID="9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"
Jul 16 21:17:59.201843 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.201826    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"} err="failed to get container status \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": rpc error: code = NotFound desc = could not find container \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": container with ID starting with 9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0 not found: ID does not exist"
Jul 16 21:17:59.201910 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.201844    2582 scope.go:117] "RemoveContainer" containerID="7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"
Jul 16 21:17:59.202056 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202039    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"} err="failed to get container status \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": rpc error: code = NotFound desc = could not find container \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": container with ID starting with 7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881 not found: ID does not exist"
Jul 16 21:17:59.202149 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202057    2582 scope.go:117] "RemoveContainer" containerID="dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"
Jul 16 21:17:59.202306 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202288    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"} err="failed to get container status \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": rpc error: code = NotFound desc = could not find container \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": container with ID starting with dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340 not found: ID does not exist"
Jul 16 21:17:59.202357 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202307    2582 scope.go:117] "RemoveContainer" containerID="d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"
Jul 16 21:17:59.202505 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202488    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"} err="failed to get container status \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": rpc error: code = NotFound desc = could not find container \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": container with ID starting with d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c not found: ID does not exist"
Jul 16 21:17:59.202572 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202507    2582 scope.go:117] "RemoveContainer" containerID="033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"
Jul 16 21:17:59.202696 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202680    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"} err="failed to get container status \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": rpc error: code = NotFound desc = could not find container \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": container with ID starting with 033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c not found: ID does not exist"
Jul 16 21:17:59.202756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202697    2582 scope.go:117] "RemoveContainer" containerID="5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"
Jul 16 21:17:59.202907 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202889    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"} err="failed to get container status \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": rpc error: code = NotFound desc = could not find container \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": container with ID starting with 5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4 not found: ID does not exist"
Jul 16 21:17:59.202967 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.202909    2582 scope.go:117] "RemoveContainer" containerID="ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"
Jul 16 21:17:59.203116 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203098    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"} err="failed to get container status \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": rpc error: code = NotFound desc = could not find container \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": container with ID starting with ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852 not found: ID does not exist"
Jul 16 21:17:59.203172 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203117    2582 scope.go:117] "RemoveContainer" containerID="9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"
Jul 16 21:17:59.203336 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203318    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"} err="failed to get container status \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": rpc error: code = NotFound desc = could not find container \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": container with ID starting with 9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0 not found: ID does not exist"
Jul 16 21:17:59.203379 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203337    2582 scope.go:117] "RemoveContainer" containerID="7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"
Jul 16 21:17:59.203543 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203527    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"} err="failed to get container status \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": rpc error: code = NotFound desc = could not find container \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": container with ID starting with 7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881 not found: ID does not exist"
Jul 16 21:17:59.203610 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203545    2582 scope.go:117] "RemoveContainer" containerID="dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"
Jul 16 21:17:59.203764 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203747    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"} err="failed to get container status \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": rpc error: code = NotFound desc = could not find container \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": container with ID starting with dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340 not found: ID does not exist"
Jul 16 21:17:59.203812 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203765    2582 scope.go:117] "RemoveContainer" containerID="d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"
Jul 16 21:17:59.203958 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203941    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"} err="failed to get container status \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": rpc error: code = NotFound desc = could not find container \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": container with ID starting with d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c not found: ID does not exist"
Jul 16 21:17:59.203997 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.203958    2582 scope.go:117] "RemoveContainer" containerID="033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"
Jul 16 21:17:59.204193 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.204175    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"} err="failed to get container status \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": rpc error: code = NotFound desc = could not find container \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": container with ID starting with 033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c not found: ID does not exist"
Jul 16 21:17:59.204260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.204194    2582 scope.go:117] "RemoveContainer" containerID="5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"
Jul 16 21:17:59.204426 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.204409    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"} err="failed to get container status \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": rpc error: code = NotFound desc = could not find container \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": container with ID starting with 5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4 not found: ID does not exist"
Jul 16 21:17:59.204472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.204427    2582 scope.go:117] "RemoveContainer" containerID="ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"
Jul 16 21:17:59.204622 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.204605    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"} err="failed to get container status \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": rpc error: code = NotFound desc = could not find container \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": container with ID starting with ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852 not found: ID does not exist"
Jul 16 21:17:59.204667 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.204622    2582 scope.go:117] "RemoveContainer" containerID="9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"
Jul 16 21:17:59.204852 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.204836    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"} err="failed to get container status \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": rpc error: code = NotFound desc = could not find container \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": container with ID starting with 9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0 not found: ID does not exist"
Jul 16 21:17:59.204896 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.204853    2582 scope.go:117] "RemoveContainer" containerID="7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"
Jul 16 21:17:59.205064 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205047    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"} err="failed to get container status \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": rpc error: code = NotFound desc = could not find container \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": container with ID starting with 7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881 not found: ID does not exist"
Jul 16 21:17:59.205136 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205064    2582 scope.go:117] "RemoveContainer" containerID="dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"
Jul 16 21:17:59.205300 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205282    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"} err="failed to get container status \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": rpc error: code = NotFound desc = could not find container \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": container with ID starting with dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340 not found: ID does not exist"
Jul 16 21:17:59.205344 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205301    2582 scope.go:117] "RemoveContainer" containerID="d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"
Jul 16 21:17:59.205513 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205492    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"} err="failed to get container status \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": rpc error: code = NotFound desc = could not find container \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": container with ID starting with d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c not found: ID does not exist"
Jul 16 21:17:59.205557 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205514    2582 scope.go:117] "RemoveContainer" containerID="033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"
Jul 16 21:17:59.205688 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205671    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"} err="failed to get container status \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": rpc error: code = NotFound desc = could not find container \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": container with ID starting with 033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c not found: ID does not exist"
Jul 16 21:17:59.205739 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205689    2582 scope.go:117] "RemoveContainer" containerID="5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"
Jul 16 21:17:59.206003 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205978    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"} err="failed to get container status \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": rpc error: code = NotFound desc = could not find container \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": container with ID starting with 5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4 not found: ID does not exist"
Jul 16 21:17:59.206003 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.205998    2582 scope.go:117] "RemoveContainer" containerID="ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"
Jul 16 21:17:59.206346 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.206278    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"} err="failed to get container status \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": rpc error: code = NotFound desc = could not find container \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": container with ID starting with ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852 not found: ID does not exist"
Jul 16 21:17:59.206346 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.206349    2582 scope.go:117] "RemoveContainer" containerID="9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"
Jul 16 21:17:59.206665 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.206639    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"} err="failed to get container status \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": rpc error: code = NotFound desc = could not find container \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": container with ID starting with 9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0 not found: ID does not exist"
Jul 16 21:17:59.206756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.206676    2582 scope.go:117] "RemoveContainer" containerID="7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"
Jul 16 21:17:59.206971 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.206944    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"} err="failed to get container status \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": rpc error: code = NotFound desc = could not find container \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": container with ID starting with 7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881 not found: ID does not exist"
Jul 16 21:17:59.206971 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.206964    2582 scope.go:117] "RemoveContainer" containerID="dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"
Jul 16 21:17:59.207338 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.207314    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"} err="failed to get container status \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": rpc error: code = NotFound desc = could not find container \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": container with ID starting with dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340 not found: ID does not exist"
Jul 16 21:17:59.207338 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.207336    2582 scope.go:117] "RemoveContainer" containerID="d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"
Jul 16 21:17:59.207537 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.207516    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"} err="failed to get container status \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": rpc error: code = NotFound desc = could not find container \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": container with ID starting with d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c not found: ID does not exist"
Jul 16 21:17:59.207601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.207539    2582 scope.go:117] "RemoveContainer" containerID="033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"
Jul 16 21:17:59.207744 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.207726    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"} err="failed to get container status \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": rpc error: code = NotFound desc = could not find container \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": container with ID starting with 033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c not found: ID does not exist"
Jul 16 21:17:59.207798 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.207743    2582 scope.go:117] "RemoveContainer" containerID="5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"
Jul 16 21:17:59.207966 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.207943    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"} err="failed to get container status \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": rpc error: code = NotFound desc = could not find container \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": container with ID starting with 5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4 not found: ID does not exist"
Jul 16 21:17:59.208061 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.207969    2582 scope.go:117] "RemoveContainer" containerID="ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"
Jul 16 21:17:59.208061 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208029    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:17:59.208206 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208187    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"} err="failed to get container status \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": rpc error: code = NotFound desc = could not find container \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": container with ID starting with ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852 not found: ID does not exist"
Jul 16 21:17:59.208206 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208206    2582 scope.go:117] "RemoveContainer" containerID="9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"
Jul 16 21:17:59.208389 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208373    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="init-config-reloader"
Jul 16 21:17:59.208448 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208394    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="init-config-reloader"
Jul 16 21:17:59.208448 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208402    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0"} err="failed to get container status \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": rpc error: code = NotFound desc = could not find container \"9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0\": container with ID starting with 9256f127cd49734c0e5c7d74d8c13d0c0707e630c0cdb263917db50a16a042d0 not found: ID does not exist"
Jul 16 21:17:59.208448 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208418    2582 scope.go:117] "RemoveContainer" containerID="7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"
Jul 16 21:17:59.208448 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208419    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="thanos-sidecar"
Jul 16 21:17:59.208448 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208430    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="thanos-sidecar"
Jul 16 21:17:59.208448 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208446    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy-web"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208455    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy-web"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208468    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="config-reloader"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208477    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="config-reloader"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208492    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="prometheus"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208500    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="prometheus"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208510    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy-thanos"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208518    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy-thanos"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208531    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208539    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208604    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="prometheus"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208616    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="thanos-sidecar"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208628    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208638    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy-web"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208646    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="config-reloader"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208655    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" containerName="kube-rbac-proxy-thanos"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208612    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881"} err="failed to get container status \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": rpc error: code = NotFound desc = could not find container \"7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881\": container with ID starting with 7147e373fb424c59ab6c4427f089546b7ecc78ec7ea9db2858e4b509ee636881 not found: ID does not exist"
Jul 16 21:17:59.208727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208700    2582 scope.go:117] "RemoveContainer" containerID="dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"
Jul 16 21:17:59.209504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208892    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340"} err="failed to get container status \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": rpc error: code = NotFound desc = could not find container \"dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340\": container with ID starting with dfcc27e77e431bb4e198145d660ae1914ea9f6bfc3610c2edbe2bd7bac39d340 not found: ID does not exist"
Jul 16 21:17:59.209504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.208906    2582 scope.go:117] "RemoveContainer" containerID="d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"
Jul 16 21:17:59.209504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.209113    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c"} err="failed to get container status \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": rpc error: code = NotFound desc = could not find container \"d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c\": container with ID starting with d31ca9ed3c20adf8b2a78fabcf00a44bdc3738409abfafca37dba47a54110e5c not found: ID does not exist"
Jul 16 21:17:59.209504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.209128    2582 scope.go:117] "RemoveContainer" containerID="033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"
Jul 16 21:17:59.209504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.209334    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c"} err="failed to get container status \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": rpc error: code = NotFound desc = could not find container \"033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c\": container with ID starting with 033914bcf8355dca18e081b1160ea97878e60a1d352dcd08c1cc54b4071aff8c not found: ID does not exist"
Jul 16 21:17:59.209504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.209349    2582 scope.go:117] "RemoveContainer" containerID="5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"
Jul 16 21:17:59.209779 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.209543    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4"} err="failed to get container status \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": rpc error: code = NotFound desc = could not find container \"5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4\": container with ID starting with 5f000166a20e3c2cfb630d7cc93f0cfbed2aef1126d070094d6329feaa3c40d4 not found: ID does not exist"
Jul 16 21:17:59.209779 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.209556    2582 scope.go:117] "RemoveContainer" containerID="ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"
Jul 16 21:17:59.209779 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.209736    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852"} err="failed to get container status \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": rpc error: code = NotFound desc = could not find container \"ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852\": container with ID starting with ad70bb4c50378e45fd85dcb5bf23842ce23a1513532046be01f7744dff97f852 not found: ID does not exist"
Jul 16 21:17:59.212734 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.212715    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.217033 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.217016    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 21:17:59.217729 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.217711    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 21:17:59.217816 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.217711    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 21:17:59.217816 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.217714    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 21:17:59.217916 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.217717    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 21:17:59.217916 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.217736    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-r6vng\""
Jul 16 21:17:59.218162 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.218065    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 21:17:59.218162 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.218155    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 21:17:59.218456 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.218428    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 21:17:59.218553 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.218499    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 21:17:59.218611 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.218559    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 21:17:59.218611 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.218569    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 21:17:59.218851 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.218836    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 21:17:59.218923 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.218907    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d22n4jl339q5l\""
Jul 16 21:17:59.218997 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.218836    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 21:17:59.221269 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.221252    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 21:17:59.239005 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.238980    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 21:17:59.247154 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.247133    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:17:59.306197 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306158    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306197 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306199    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306411 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306225    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306411 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306269    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306411 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306293    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306411 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306328    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306411 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306367    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306429    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51fea4e7-4c47-40c7-9299-8b4351c1bf34-config-out\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306449    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9nlc\" (UniqueName: \"kubernetes.io/projected/51fea4e7-4c47-40c7-9299-8b4351c1bf34-kube-api-access-v9nlc\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306474    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306498    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-config\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51fea4e7-4c47-40c7-9299-8b4351c1bf34-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306555    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306579    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306618    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-web-config\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306638    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306878 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306715    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.306878 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.306735    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407120    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9nlc\" (UniqueName: \"kubernetes.io/projected/51fea4e7-4c47-40c7-9299-8b4351c1bf34-kube-api-access-v9nlc\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407163    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-config\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407493 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407302    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51fea4e7-4c47-40c7-9299-8b4351c1bf34-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407493 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407493 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407493 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407400    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-web-config\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407493 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407428    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407493 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407453    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407537    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407570    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407597    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407622    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407670    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.407784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407771    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.408405 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51fea4e7-4c47-40c7-9299-8b4351c1bf34-config-out\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.408405 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.407995    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.408405 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.408030    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.408986 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.408637    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.408986 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.408704    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.409339 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.409312    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.410268 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.410233    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.410649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.410625    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-config\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.410840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.410819    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51fea4e7-4c47-40c7-9299-8b4351c1bf34-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.411298 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.411269    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.411391 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.411272    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.412658 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.411926    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.412658 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.412417    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51fea4e7-4c47-40c7-9299-8b4351c1bf34-config-out\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.412658 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.412607    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.412658 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.412618    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.412847 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.412667    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-web-config\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.412847 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.412675    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/51fea4e7-4c47-40c7-9299-8b4351c1bf34-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.413842 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.413821    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.414029 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.414011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.414324 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.414305    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51fea4e7-4c47-40c7-9299-8b4351c1bf34-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.417502 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.417481    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9nlc\" (UniqueName: \"kubernetes.io/projected/51fea4e7-4c47-40c7-9299-8b4351c1bf34-kube-api-access-v9nlc\") pod \"prometheus-k8s-0\" (UID: \"51fea4e7-4c47-40c7-9299-8b4351c1bf34\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.522229 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.522190    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:17:59.668358 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:17:59.668308    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:17:59.670797 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:17:59.670765    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51fea4e7_4c47_40c7_9299_8b4351c1bf34.slice/crio-1d90bb7fe9486cd5d304d445706243fd9e99ec851a3133c07e8ba58ececb6c25 WatchSource:0}: Error finding container 1d90bb7fe9486cd5d304d445706243fd9e99ec851a3133c07e8ba58ececb6c25: Status 404 returned error can't find the container with id 1d90bb7fe9486cd5d304d445706243fd9e99ec851a3133c07e8ba58ececb6c25
Jul 16 21:18:00.148495 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:00.148456    2582 generic.go:356] "Generic (PLEG): container finished" podID="51fea4e7-4c47-40c7-9299-8b4351c1bf34" containerID="67a6a19b441ee68fb22e58b25b3300814eb55e41e66461ebec2ed45fb9013054" exitCode=0
Jul 16 21:18:00.148664 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:00.148507    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51fea4e7-4c47-40c7-9299-8b4351c1bf34","Type":"ContainerDied","Data":"67a6a19b441ee68fb22e58b25b3300814eb55e41e66461ebec2ed45fb9013054"}
Jul 16 21:18:00.148664 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:00.148571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51fea4e7-4c47-40c7-9299-8b4351c1bf34","Type":"ContainerStarted","Data":"1d90bb7fe9486cd5d304d445706243fd9e99ec851a3133c07e8ba58ececb6c25"}
Jul 16 21:18:00.505820 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:00.505791    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a408009-5837-4ceb-8aa8-93b044b85b40" path="/var/lib/kubelet/pods/7a408009-5837-4ceb-8aa8-93b044b85b40/volumes"
Jul 16 21:18:01.155847 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:01.155817    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51fea4e7-4c47-40c7-9299-8b4351c1bf34","Type":"ContainerStarted","Data":"cc0246a37586cc3ee20cfebc7708b44ab2523de38fe9fab7c392edd9137cc8ad"}
Jul 16 21:18:01.156053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:01.155886    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51fea4e7-4c47-40c7-9299-8b4351c1bf34","Type":"ContainerStarted","Data":"0001e3713a7b5acc04bb91cf7c2a45ca786d40cab469c391cdf5d303eef40156"}
Jul 16 21:18:01.156053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:01.155913    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51fea4e7-4c47-40c7-9299-8b4351c1bf34","Type":"ContainerStarted","Data":"d59ddac8a1329d72363369bb5220f40039982ca797489076fb69186960a6fcf5"}
Jul 16 21:18:01.156053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:01.155936    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51fea4e7-4c47-40c7-9299-8b4351c1bf34","Type":"ContainerStarted","Data":"eef1a19c5f3ae1cd61ecaa323af94f68abbe82c7b172ceb07d512d7127571ac5"}
Jul 16 21:18:01.156053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:01.155959    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51fea4e7-4c47-40c7-9299-8b4351c1bf34","Type":"ContainerStarted","Data":"dc9126a783f7b15cf0c0e61bd0369be1fb037475644337ac86c8436ce0aae709"}
Jul 16 21:18:01.156053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:01.155989    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51fea4e7-4c47-40c7-9299-8b4351c1bf34","Type":"ContainerStarted","Data":"261aaa35ea99450550e7bcf538d814b00fa833da4074e0e8334c4245635bc198"}
Jul 16 21:18:01.205495 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:01.205442    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.186346296 podStartE2EDuration="2.205426755s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.080459ms" isStatefulPod=false podCreationTimestamp="2026-07-16 21:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:18:01.203776041 +0000 UTC m=+145.260606973" watchObservedRunningTime="2026-07-16 21:18:01.205426755 +0000 UTC m=+145.262257667"
Jul 16 21:18:04.104544 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:04.104512    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:18:04.198120 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:04.198071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:18:04.523204 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:04.523116    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:18:15.482855 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:15.482815    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:18:15.571839 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:15.571778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:18:15.654116 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:15.654063    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:18:22.011350 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:22.011316    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:18:22.090930 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:22.090897    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:18:33.513996 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:33.513892    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:18:58.579711 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:58.579666    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:18:59.523213 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:59.523182    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:18:59.538875 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:18:59.538853    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:19:00.383872 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:00.383846    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:19:05.447114 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:05.447067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:19:07.973448 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:07.973411    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:19:08.048317 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:08.048287    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:19:19.924175 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:19.924140    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:19:19.983212 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:19.983182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:19:34.808206 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:34.808172    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:19:34.909698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:34.909661    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:19:51.235502 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:51.235469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:19:51.297244 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:19:51.297207    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:04.327929 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:04.327860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:06.524681 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:06.524650    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:15.741249 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:15.741219    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:15.844160 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:15.844122    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:15.918224 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:15.918189    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:17.813470 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:17.813441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:18.836064 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:18.836035    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:21.435125 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:21.435093    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:21.537344 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:21.537309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:22.960529 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:22.960494    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:23.068274 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:23.068237    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:23.327933 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:23.327893    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:23.428296 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:23.428214    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:25.730671 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:25.730633    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:26.155894 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:26.155851    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:27.764201 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:27.764166    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:28.247229 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:28.247192    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:28.314818 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:28.314784    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:33.315170 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:33.315138    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:35.906218 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:35.906182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:36.005326 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:36.005296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:36.325939 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:36.325910    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 21:20:37.274562 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.274530    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"]
Jul 16 21:20:37.276926 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.276911    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"
Jul 16 21:20:37.279144 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.279123    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 16 21:20:37.279814 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.279799    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:20:37.279873 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.279828    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-n9hh2\""
Jul 16 21:20:37.285629 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.285605    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"]
Jul 16 21:20:37.344063 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.344020    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4fad4a4-41a4-46de-a2eb-0ba9783a8b89-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rz7d4\" (UID: \"c4fad4a4-41a4-46de-a2eb-0ba9783a8b89\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"
Jul 16 21:20:37.344063 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.344067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbtp4\" (UniqueName: \"kubernetes.io/projected/c4fad4a4-41a4-46de-a2eb-0ba9783a8b89-kube-api-access-rbtp4\") pod \"openshift-lws-operator-fd8ccff4c-rz7d4\" (UID: \"c4fad4a4-41a4-46de-a2eb-0ba9783a8b89\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"
Jul 16 21:20:37.445359 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.445318    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4fad4a4-41a4-46de-a2eb-0ba9783a8b89-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rz7d4\" (UID: \"c4fad4a4-41a4-46de-a2eb-0ba9783a8b89\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"
Jul 16 21:20:37.445542 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.445370    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbtp4\" (UniqueName: \"kubernetes.io/projected/c4fad4a4-41a4-46de-a2eb-0ba9783a8b89-kube-api-access-rbtp4\") pod \"openshift-lws-operator-fd8ccff4c-rz7d4\" (UID: \"c4fad4a4-41a4-46de-a2eb-0ba9783a8b89\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"
Jul 16 21:20:37.445726 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.445704    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4fad4a4-41a4-46de-a2eb-0ba9783a8b89-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rz7d4\" (UID: \"c4fad4a4-41a4-46de-a2eb-0ba9783a8b89\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"
Jul 16 21:20:37.454306 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.454282    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbtp4\" (UniqueName: \"kubernetes.io/projected/c4fad4a4-41a4-46de-a2eb-0ba9783a8b89-kube-api-access-rbtp4\") pod \"openshift-lws-operator-fd8ccff4c-rz7d4\" (UID: \"c4fad4a4-41a4-46de-a2eb-0ba9783a8b89\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"
Jul 16 21:20:37.593745 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.593666    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"
Jul 16 21:20:37.724823 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.724795    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4"]
Jul 16 21:20:37.727504 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:20:37.727479    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4fad4a4_41a4_46de_a2eb_0ba9783a8b89.slice/crio-af5e62cd2fe416fc0439e4fe21fd6911be6a65cfa9ce3daf3d5a8afc367a216d WatchSource:0}: Error finding container af5e62cd2fe416fc0439e4fe21fd6911be6a65cfa9ce3daf3d5a8afc367a216d: Status 404 returned error can't find the container with id af5e62cd2fe416fc0439e4fe21fd6911be6a65cfa9ce3daf3d5a8afc367a216d
Jul 16 21:20:37.729360 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:37.729342    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:20:38.697034 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:38.697004    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4" event={"ID":"c4fad4a4-41a4-46de-a2eb-0ba9783a8b89","Type":"ContainerStarted","Data":"af5e62cd2fe416fc0439e4fe21fd6911be6a65cfa9ce3daf3d5a8afc367a216d"}
Jul 16 21:20:40.705968 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:40.705871    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4" event={"ID":"c4fad4a4-41a4-46de-a2eb-0ba9783a8b89","Type":"ContainerStarted","Data":"ac8da2d146854c93795d42d207e5e8de6abec1394077f95e6e1542780fe48693"}
Jul 16 21:20:41.432690 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:41.432666    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:41.538340 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:41.538310    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:43.507433 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:43.507397    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:57.396748 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:57.396714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:57.497658 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:57.497623    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:59.253788 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:59.253758    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:20:59.360620 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:20:59.360586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:00.062044 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.062007    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:00.260710 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.260672    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:00.372792 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.372704    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:00.396907 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.396861    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rz7d4" podStartSLOduration=20.728212052 podStartE2EDuration="23.396846525s" totalImagesPullingTime="2.668634473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:20:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:20:40.72445163 +0000 UTC m=+304.781282542" watchObservedRunningTime="2026-07-16 21:21:00.396846525 +0000 UTC m=+324.453677436"
Jul 16 21:21:00.397882 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.397863    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"]
Jul 16 21:21:00.401782 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.401764    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.405311 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.405290    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 21:21:00.405436 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.405290    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 21:21:00.406544 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.406522    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 21:21:00.406618 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.406575    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 21:21:00.406664 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.406536    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xg8s6\""
Jul 16 21:21:00.424620 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.424597    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"]
Jul 16 21:21:00.541317 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.541285    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/255465d7-fbd4-4536-baf0-2a1b696f322f-manifests\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.541317 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.541317    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/255465d7-fbd4-4536-baf0-2a1b696f322f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.541512 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.541353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zscq\" (UniqueName: \"kubernetes.io/projected/255465d7-fbd4-4536-baf0-2a1b696f322f-kube-api-access-8zscq\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.541512 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.541450    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/255465d7-fbd4-4536-baf0-2a1b696f322f-tmp\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.541512 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.541486    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/255465d7-fbd4-4536-baf0-2a1b696f322f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.548803 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.548779    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:00.642935 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.642851    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/255465d7-fbd4-4536-baf0-2a1b696f322f-manifests\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.642935 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.642886    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/255465d7-fbd4-4536-baf0-2a1b696f322f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.642935 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.642920    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zscq\" (UniqueName: \"kubernetes.io/projected/255465d7-fbd4-4536-baf0-2a1b696f322f-kube-api-access-8zscq\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.643240 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.642958    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/255465d7-fbd4-4536-baf0-2a1b696f322f-tmp\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.643240 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.642981    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/255465d7-fbd4-4536-baf0-2a1b696f322f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.643382 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.643362    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/255465d7-fbd4-4536-baf0-2a1b696f322f-manifests\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.645399 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.645379    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/255465d7-fbd4-4536-baf0-2a1b696f322f-tmp\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.645478 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.645426    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/255465d7-fbd4-4536-baf0-2a1b696f322f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.645529 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.645513    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/255465d7-fbd4-4536-baf0-2a1b696f322f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.649644 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.649620    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:00.651906 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.651890    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zscq\" (UniqueName: \"kubernetes.io/projected/255465d7-fbd4-4536-baf0-2a1b696f322f-kube-api-access-8zscq\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2\" (UID: \"255465d7-fbd4-4536-baf0-2a1b696f322f\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.711447 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.711416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:00.749741 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.749716    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:00.847828 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.847797    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"]
Jul 16 21:21:00.850833 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:21:00.850802    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod255465d7_fbd4_4536_baf0_2a1b696f322f.slice/crio-e9a3911ee718c49f1453714d2594b87b2bc808c72310456a763318af2937ae5f WatchSource:0}: Error finding container e9a3911ee718c49f1453714d2594b87b2bc808c72310456a763318af2937ae5f: Status 404 returned error can't find the container with id e9a3911ee718c49f1453714d2594b87b2bc808c72310456a763318af2937ae5f
Jul 16 21:21:00.851363 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.851336    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:00.957310 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:00.957223    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:01.785904 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:01.785864    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2" event={"ID":"255465d7-fbd4-4536-baf0-2a1b696f322f","Type":"ContainerStarted","Data":"e9a3911ee718c49f1453714d2594b87b2bc808c72310456a763318af2937ae5f"}
Jul 16 21:21:04.798643 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:04.798607    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2" event={"ID":"255465d7-fbd4-4536-baf0-2a1b696f322f","Type":"ContainerStarted","Data":"73a135d7a5973aca20edcbf6421ecdae5a27d9aad53613f84d840d3194eb4614"}
Jul 16 21:21:05.803725 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:05.803687    2582 generic.go:356] "Generic (PLEG): container finished" podID="255465d7-fbd4-4536-baf0-2a1b696f322f" containerID="73a135d7a5973aca20edcbf6421ecdae5a27d9aad53613f84d840d3194eb4614" exitCode=0
Jul 16 21:21:05.804128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:05.803755    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2" event={"ID":"255465d7-fbd4-4536-baf0-2a1b696f322f","Type":"ContainerDied","Data":"73a135d7a5973aca20edcbf6421ecdae5a27d9aad53613f84d840d3194eb4614"}
Jul 16 21:21:06.809160 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:06.809126    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2" event={"ID":"255465d7-fbd4-4536-baf0-2a1b696f322f","Type":"ContainerStarted","Data":"451684e0a122dbf7296ac1c330ef8573f14a1f381e62d52ea9725b93fdadf415"}
Jul 16 21:21:06.809551 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:06.809231    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:06.834963 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:06.834878    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2" podStartSLOduration=2.554306686 podStartE2EDuration="6.834858749s" totalImagesPullingTime="3.250216757s" totalInitContainerRuntime="1.030335306s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:06.833250779 +0000 UTC m=+330.890081688" watchObservedRunningTime="2026-07-16 21:21:06.834858749 +0000 UTC m=+330.891689657"
Jul 16 21:21:07.826336 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:07.826278    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:07.926770 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:07.926721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:11.998037 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:11.997992    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:12.102072 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:12.102039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:17.815035 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:17.815007    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2"
Jul 16 21:21:20.430164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:20.430128    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:20.543304 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:20.543269    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:21.114541 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.114511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:21.213115 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.213050    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:21.270621 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.270590    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"]
Jul 16 21:21:21.277639 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.277614    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.283858 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.283837    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 21:21:21.283970 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.283888    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 16 21:21:21.284057 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.284034    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 16 21:21:21.284188 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.284103    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 21:21:21.284188 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.284108    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-f6rp2\""
Jul 16 21:21:21.301331 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.301307    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"]
Jul 16 21:21:21.325956 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.325924    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2e98803-3f5b-4e4b-b04f-9adfd5045a74-tmp\") pod \"kube-auth-proxy-796f7d4c6-wjjfl\" (UID: \"d2e98803-3f5b-4e4b-b04f-9adfd5045a74\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.326139 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.325969    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56gmr\" (UniqueName: \"kubernetes.io/projected/d2e98803-3f5b-4e4b-b04f-9adfd5045a74-kube-api-access-56gmr\") pod \"kube-auth-proxy-796f7d4c6-wjjfl\" (UID: \"d2e98803-3f5b-4e4b-b04f-9adfd5045a74\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.326139 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.326040    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2e98803-3f5b-4e4b-b04f-9adfd5045a74-tls-certs\") pod \"kube-auth-proxy-796f7d4c6-wjjfl\" (UID: \"d2e98803-3f5b-4e4b-b04f-9adfd5045a74\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.426870 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.426784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2e98803-3f5b-4e4b-b04f-9adfd5045a74-tls-certs\") pod \"kube-auth-proxy-796f7d4c6-wjjfl\" (UID: \"d2e98803-3f5b-4e4b-b04f-9adfd5045a74\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.426870 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.426849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2e98803-3f5b-4e4b-b04f-9adfd5045a74-tmp\") pod \"kube-auth-proxy-796f7d4c6-wjjfl\" (UID: \"d2e98803-3f5b-4e4b-b04f-9adfd5045a74\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.427100 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.427023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56gmr\" (UniqueName: \"kubernetes.io/projected/d2e98803-3f5b-4e4b-b04f-9adfd5045a74-kube-api-access-56gmr\") pod \"kube-auth-proxy-796f7d4c6-wjjfl\" (UID: \"d2e98803-3f5b-4e4b-b04f-9adfd5045a74\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.429156 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.429128    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2e98803-3f5b-4e4b-b04f-9adfd5045a74-tmp\") pod \"kube-auth-proxy-796f7d4c6-wjjfl\" (UID: \"d2e98803-3f5b-4e4b-b04f-9adfd5045a74\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.429435 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.429407    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2e98803-3f5b-4e4b-b04f-9adfd5045a74-tls-certs\") pod \"kube-auth-proxy-796f7d4c6-wjjfl\" (UID: \"d2e98803-3f5b-4e4b-b04f-9adfd5045a74\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.430015 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.429998    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:21.435732 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.435701    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56gmr\" (UniqueName: \"kubernetes.io/projected/d2e98803-3f5b-4e4b-b04f-9adfd5045a74-kube-api-access-56gmr\") pod \"kube-auth-proxy-796f7d4c6-wjjfl\" (UID: \"d2e98803-3f5b-4e4b-b04f-9adfd5045a74\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.500332 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.500300    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:21.586379 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.586342    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"
Jul 16 21:21:21.609614 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.609590    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:21.731165 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.731068    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl"]
Jul 16 21:21:21.735032 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:21:21.735002    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e98803_3f5b_4e4b_b04f_9adfd5045a74.slice/crio-86d27e6fb6f9181bddaa1a040c196903f9bd01c74483afd59f5cf455999f88ec WatchSource:0}: Error finding container 86d27e6fb6f9181bddaa1a040c196903f9bd01c74483afd59f5cf455999f88ec: Status 404 returned error can't find the container with id 86d27e6fb6f9181bddaa1a040c196903f9bd01c74483afd59f5cf455999f88ec
Jul 16 21:21:21.738644 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.738622    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:21.861527 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.861493    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl" event={"ID":"d2e98803-3f5b-4e4b-b04f-9adfd5045a74","Type":"ContainerStarted","Data":"86d27e6fb6f9181bddaa1a040c196903f9bd01c74483afd59f5cf455999f88ec"}
Jul 16 21:21:21.936334 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:21.936299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:22.195536 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.195505    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:22.201420 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.201397    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"]
Jul 16 21:21:22.204560 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.204544    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.207088 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.207055    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 16 21:21:22.207174 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.207103    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-72vp7\""
Jul 16 21:21:22.207575 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.207558    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 21:21:22.207638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.207594    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 16 21:21:22.207695 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.207679    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 16 21:21:22.222454 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.222423    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"]
Jul 16 21:21:22.231317 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.231296    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-cacerts\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.231421 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.231349    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.231421 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.231387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbfvj\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-kube-api-access-cbfvj\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.231486 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.231438    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.231486 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.231454    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-token\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.231486 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.231483    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-local-certs\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.231601 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.231499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.294175 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.294141    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:22.332108 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.332049    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-cacerts\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.332301 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.332143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.332301 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.332173    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbfvj\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-kube-api-access-cbfvj\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.332301 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.332206    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.332301 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.332227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-token\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.332301 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.332266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-local-certs\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.332301 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.332289    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.332788 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.332762    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.334645 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.334622    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.334900 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.334886    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-local-certs\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.335157 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.335140    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.335238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.335176    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-cacerts\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.343409 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.343380    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbfvj\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-kube-api-access-cbfvj\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.343559 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.343542    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-token\") pod \"istiod-openshift-gateway-7574459656-ddst2\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.513629 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.513542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:22.514344 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.514300    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:22.674410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.674381    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"]
Jul 16 21:21:22.676664 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:21:22.676630    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea76332_1f07_4868_9ea4_4d3ce9e869b5.slice/crio-833408486d0aa343f325898b59f367a11a288e72aa33febbd5fc01162ad709e4 WatchSource:0}: Error finding container 833408486d0aa343f325898b59f367a11a288e72aa33febbd5fc01162ad709e4: Status 404 returned error can't find the container with id 833408486d0aa343f325898b59f367a11a288e72aa33febbd5fc01162ad709e4
Jul 16 21:21:22.688228 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.688200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:22.867371 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:22.867332    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2" event={"ID":"7ea76332-1f07-4868-9ea4-4d3ce9e869b5","Type":"ContainerStarted","Data":"833408486d0aa343f325898b59f367a11a288e72aa33febbd5fc01162ad709e4"}
Jul 16 21:21:23.265598 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:23.265514    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:25.356911 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:25.356875    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 16 21:21:25.357260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:25.356958    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 16 21:21:25.881027 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:25.880914    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2" event={"ID":"7ea76332-1f07-4868-9ea4-4d3ce9e869b5","Type":"ContainerStarted","Data":"74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191"}
Jul 16 21:21:25.881027 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:25.881020    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:25.882458 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:25.882430    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl" event={"ID":"d2e98803-3f5b-4e4b-b04f-9adfd5045a74","Type":"ContainerStarted","Data":"5330b6e37bb992924b089f7286ef0cd021ce300897605cd8b25b5254e28ff59a"}
Jul 16 21:21:25.902640 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:25.902589    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2" podStartSLOduration=1.2252692920000001 podStartE2EDuration="3.902573344s" totalImagesPullingTime="2.677304052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:25.900527423 +0000 UTC m=+349.957358334" watchObservedRunningTime="2026-07-16 21:21:25.902573344 +0000 UTC m=+349.959404233"
Jul 16 21:21:25.916974 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:25.916918    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-wjjfl" podStartSLOduration=1.32987937 podStartE2EDuration="4.916901171s" totalImagesPullingTime="3.587021801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:25.916642846 +0000 UTC m=+349.973473759" watchObservedRunningTime="2026-07-16 21:21:25.916901171 +0000 UTC m=+349.973732116"
Jul 16 21:21:26.171170 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:26.171066    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:26.269056 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:26.269005    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:26.876151 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:26.876103    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:26.887649 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:26.887606    2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-ddst2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:21:26.887790 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:26.887685    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2" podUID="7ea76332-1f07-4868-9ea4-4d3ce9e869b5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:21:27.092332 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:27.092296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:27.435308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:27.435272    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:29.354270 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.354236    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"]
Jul 16 21:21:29.356584 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.356566    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.366832 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.366815    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 16 21:21:29.367674 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.367655    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 16 21:21:29.367768 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.367691    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 16 21:21:29.367768 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.367690    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wk2pn\""
Jul 16 21:21:29.377190 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.377166    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"]
Jul 16 21:21:29.394962 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.394938    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/707c0999-19c7-45b6-8e20-6076dad3bb50-manager-config\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.395065 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.394969    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9bqd\" (UniqueName: \"kubernetes.io/projected/707c0999-19c7-45b6-8e20-6076dad3bb50-kube-api-access-k9bqd\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.395065 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.394994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/707c0999-19c7-45b6-8e20-6076dad3bb50-cert\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.395176 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.395092    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/707c0999-19c7-45b6-8e20-6076dad3bb50-metrics-cert\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.495865 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.495837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/707c0999-19c7-45b6-8e20-6076dad3bb50-manager-config\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.495865 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.495870    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9bqd\" (UniqueName: \"kubernetes.io/projected/707c0999-19c7-45b6-8e20-6076dad3bb50-kube-api-access-k9bqd\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.496096 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.495888    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/707c0999-19c7-45b6-8e20-6076dad3bb50-cert\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.496096 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.495905    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/707c0999-19c7-45b6-8e20-6076dad3bb50-metrics-cert\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.496496 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.496473    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/707c0999-19c7-45b6-8e20-6076dad3bb50-manager-config\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.498395 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.498374    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/707c0999-19c7-45b6-8e20-6076dad3bb50-metrics-cert\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.498547 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.498527    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/707c0999-19c7-45b6-8e20-6076dad3bb50-cert\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.512498 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.512470    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9bqd\" (UniqueName: \"kubernetes.io/projected/707c0999-19c7-45b6-8e20-6076dad3bb50-kube-api-access-k9bqd\") pod \"lws-controller-manager-78cbff6c4-kcpzw\" (UID: \"707c0999-19c7-45b6-8e20-6076dad3bb50\") " pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.664815 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.664728    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:29.817973 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:21:29.817937    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707c0999_19c7_45b6_8e20_6076dad3bb50.slice/crio-1925e115660c1ecf94a96133112a6966dffd38607a9734d02ac5c44e6949d4a7 WatchSource:0}: Error finding container 1925e115660c1ecf94a96133112a6966dffd38607a9734d02ac5c44e6949d4a7: Status 404 returned error can't find the container with id 1925e115660c1ecf94a96133112a6966dffd38607a9734d02ac5c44e6949d4a7
Jul 16 21:21:29.825529 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.825507    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"]
Jul 16 21:21:29.887037 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.887008    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:29.897602 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:29.897580    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw" event={"ID":"707c0999-19c7-45b6-8e20-6076dad3bb50","Type":"ContainerStarted","Data":"1925e115660c1ecf94a96133112a6966dffd38607a9734d02ac5c44e6949d4a7"}
Jul 16 21:21:31.877675 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:31.877643    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:31.907298 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:31.907268    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw" event={"ID":"707c0999-19c7-45b6-8e20-6076dad3bb50","Type":"ContainerStarted","Data":"757d363cb079405c3b09f20258fa4ca3d3e7491583e0df882c53d6e0ca3f580f"}
Jul 16 21:21:31.907459 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:31.907386    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:21:31.934271 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:31.934213    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw" podStartSLOduration=1.457383835 podStartE2EDuration="2.934198728s" totalImagesPullingTime="1.476814893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:31.932522278 +0000 UTC m=+355.989353190" watchObservedRunningTime="2026-07-16 21:21:31.934198728 +0000 UTC m=+355.991029638"
Jul 16 21:21:32.031850 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:32.031818    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:32.314813 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:32.314783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:32.506253 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:32.506227    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:33.200437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:33.200389    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:33.295987 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:33.295955    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:33.679616 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:33.679564    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:21:34.071229 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.071189    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"]
Jul 16 21:21:34.071437 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.071408    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2" podUID="7ea76332-1f07-4868-9ea4-4d3ce9e869b5" containerName="discovery" containerID="cri-o://74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191" gracePeriod=30
Jul 16 21:21:34.325235 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.325183    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:34.443011 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.442976    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-kubeconfig\") pod \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") "
Jul 16 21:21:34.443011 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.443019    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-token\") pod \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") "
Jul 16 21:21:34.443274 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.443036    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-dns-cert\") pod \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") "
Jul 16 21:21:34.443274 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.443066    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-local-certs\") pod \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") "
Jul 16 21:21:34.443274 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.443121    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-ca-configmap\") pod \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") "
Jul 16 21:21:34.443274 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.443140    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-cacerts\") pod \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") "
Jul 16 21:21:34.443274 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.443154    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbfvj\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-kube-api-access-cbfvj\") pod \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\" (UID: \"7ea76332-1f07-4868-9ea4-4d3ce9e869b5\") "
Jul 16 21:21:34.443532 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.443503    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7ea76332-1f07-4868-9ea4-4d3ce9e869b5" (UID: "7ea76332-1f07-4868-9ea4-4d3ce9e869b5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:21:34.445574 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.445545    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7ea76332-1f07-4868-9ea4-4d3ce9e869b5" (UID: "7ea76332-1f07-4868-9ea4-4d3ce9e869b5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:21:34.445712 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.445577    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-kube-api-access-cbfvj" (OuterVolumeSpecName: "kube-api-access-cbfvj") pod "7ea76332-1f07-4868-9ea4-4d3ce9e869b5" (UID: "7ea76332-1f07-4868-9ea4-4d3ce9e869b5"). InnerVolumeSpecName "kube-api-access-cbfvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:21:34.445844 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.445813    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7ea76332-1f07-4868-9ea4-4d3ce9e869b5" (UID: "7ea76332-1f07-4868-9ea4-4d3ce9e869b5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:21:34.445942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.445860    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-local-certs" (OuterVolumeSpecName: "local-certs") pod "7ea76332-1f07-4868-9ea4-4d3ce9e869b5" (UID: "7ea76332-1f07-4868-9ea4-4d3ce9e869b5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:21:34.445942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.445901    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-cacerts" (OuterVolumeSpecName: "cacerts") pod "7ea76332-1f07-4868-9ea4-4d3ce9e869b5" (UID: "7ea76332-1f07-4868-9ea4-4d3ce9e869b5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:21:34.445942 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.445917    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ea76332-1f07-4868-9ea4-4d3ce9e869b5" (UID: "7ea76332-1f07-4868-9ea4-4d3ce9e869b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:21:34.544164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.544135    2582 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-kubeconfig\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:21:34.544164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.544157    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-token\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:21:34.544164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.544167    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-dns-cert\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:21:34.544359 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.544176    2582 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-local-certs\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:21:34.544359 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.544184    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-istio-csr-ca-configmap\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:21:34.544359 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.544193    2582 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-cacerts\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:21:34.544359 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.544202    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbfvj\" (UniqueName: \"kubernetes.io/projected/7ea76332-1f07-4868-9ea4-4d3ce9e869b5-kube-api-access-cbfvj\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:21:34.918775 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.918740    2582 generic.go:356] "Generic (PLEG): container finished" podID="7ea76332-1f07-4868-9ea4-4d3ce9e869b5" containerID="74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191" exitCode=0
Jul 16 21:21:34.918927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.918791    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"
Jul 16 21:21:34.918927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.918818    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2" event={"ID":"7ea76332-1f07-4868-9ea4-4d3ce9e869b5","Type":"ContainerDied","Data":"74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191"}
Jul 16 21:21:34.918927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.918862    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ddst2" event={"ID":"7ea76332-1f07-4868-9ea4-4d3ce9e869b5","Type":"ContainerDied","Data":"833408486d0aa343f325898b59f367a11a288e72aa33febbd5fc01162ad709e4"}
Jul 16 21:21:34.918927 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.918881    2582 scope.go:117] "RemoveContainer" containerID="74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191"
Jul 16 21:21:34.926966 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.926950    2582 scope.go:117] "RemoveContainer" containerID="74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191"
Jul 16 21:21:34.927238 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:21:34.927220    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191\": container with ID starting with 74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191 not found: ID does not exist" containerID="74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191"
Jul 16 21:21:34.927304 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.927247    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191"} err="failed to get container status \"74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191\": rpc error: code = NotFound desc = could not find container \"74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191\": container with ID starting with 74804c603f048680f5daf5e91fd7853cac75047ce5a01cf537690bdc17032191 not found: ID does not exist"
Jul 16 21:21:34.955487 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.955464    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"]
Jul 16 21:21:34.965509 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:34.965483    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ddst2"]
Jul 16 21:21:36.504914 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:36.504874    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ea76332-1f07-4868-9ea4-4d3ce9e869b5" path="/var/lib/kubelet/pods/7ea76332-1f07-4868-9ea4-4d3ce9e869b5/volumes"
Jul 16 21:21:42.913510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:21:42.913477    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78cbff6c4-kcpzw"
Jul 16 21:22:21.876067 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:21.876022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:22:21.975804 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:21.975776    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:22:22.059329 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:22.059295    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:22:22.873193 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:22.873147    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:22:22.971623 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:22.971582    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:22:50.271473 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:50.271426    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:22:53.264807 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:53.264775    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:22:54.386260 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:54.386225    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:22:58.113214 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:58.113124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:22:58.341011 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:22:58.340973    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:00.312894 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.312864    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"]
Jul 16 21:23:00.313290 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.313151    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea76332-1f07-4868-9ea4-4d3ce9e869b5" containerName="discovery"
Jul 16 21:23:00.313290 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.313162    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea76332-1f07-4868-9ea4-4d3ce9e869b5" containerName="discovery"
Jul 16 21:23:00.313290 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.313215    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea76332-1f07-4868-9ea4-4d3ce9e869b5" containerName="discovery"
Jul 16 21:23:00.315136 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.315120    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"
Jul 16 21:23:00.318185 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.318166    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 16 21:23:00.318305 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.318195    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-bp4g9\""
Jul 16 21:23:00.318305 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.318230    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 21:23:00.322624 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.322605    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 21:23:00.332185 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.332166    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"]
Jul 16 21:23:00.365890 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.365850    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:00.374453 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.374427    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmqwg\" (UniqueName: \"kubernetes.io/projected/0749f338-d140-47b4-b636-9df5ae09c1c5-kube-api-access-xmqwg\") pod \"dns-operator-controller-manager-864dd78c4c-b9fqb\" (UID: \"0749f338-d140-47b4-b636-9df5ae09c1c5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"
Jul 16 21:23:00.475918 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.475883    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmqwg\" (UniqueName: \"kubernetes.io/projected/0749f338-d140-47b4-b636-9df5ae09c1c5-kube-api-access-xmqwg\") pod \"dns-operator-controller-manager-864dd78c4c-b9fqb\" (UID: \"0749f338-d140-47b4-b636-9df5ae09c1c5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"
Jul 16 21:23:00.486536 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.486507    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmqwg\" (UniqueName: \"kubernetes.io/projected/0749f338-d140-47b4-b636-9df5ae09c1c5-kube-api-access-xmqwg\") pod \"dns-operator-controller-manager-864dd78c4c-b9fqb\" (UID: \"0749f338-d140-47b4-b636-9df5ae09c1c5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"
Jul 16 21:23:00.624635 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.624550    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"
Jul 16 21:23:00.772845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.772823    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"]
Jul 16 21:23:00.775657 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:23:00.775632    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0749f338_d140_47b4_b636_9df5ae09c1c5.slice/crio-de1bb9e56f7542fff0b1f2bfcf5867b1150095a5496f3b7bf3c01680b420ec4d WatchSource:0}: Error finding container de1bb9e56f7542fff0b1f2bfcf5867b1150095a5496f3b7bf3c01680b420ec4d: Status 404 returned error can't find the container with id de1bb9e56f7542fff0b1f2bfcf5867b1150095a5496f3b7bf3c01680b420ec4d
Jul 16 21:23:00.881280 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:00.881190    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:01.241929 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:01.241846    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb" event={"ID":"0749f338-d140-47b4-b636-9df5ae09c1c5","Type":"ContainerStarted","Data":"de1bb9e56f7542fff0b1f2bfcf5867b1150095a5496f3b7bf3c01680b420ec4d"}
Jul 16 21:23:01.407830 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:01.407799    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:03.252380 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:03.252298    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb" event={"ID":"0749f338-d140-47b4-b636-9df5ae09c1c5","Type":"ContainerStarted","Data":"4ce9cfa1eee041e1931a69d99166cb8f97003ddce50ca4881e79dbccaff88b8b"}
Jul 16 21:23:03.252727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:03.252430    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"
Jul 16 21:23:03.275875 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:03.275826    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb" podStartSLOduration=1.21545879 podStartE2EDuration="3.275811532s" totalImagesPullingTime="2.060352742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:23:03.274645998 +0000 UTC m=+447.331476921" watchObservedRunningTime="2026-07-16 21:23:03.275811532 +0000 UTC m=+447.332642442"
Jul 16 21:23:04.982972 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.982937    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58cb7f5466-6jqf9"]
Jul 16 21:23:04.986018 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.986001    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:04.991699 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.991674    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 21:23:04.992427 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.992173    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 21:23:04.992567 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.992542    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 21:23:04.992567 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.992560    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 21:23:04.992567 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.992569    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 21:23:04.992788 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.992602    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 21:23:04.992788 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.992551    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ht7k4\""
Jul 16 21:23:04.992994 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.992981    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 21:23:04.998565 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:04.998548    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 21:23:05.001674 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.001656    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58cb7f5466-6jqf9"]
Jul 16 21:23:05.055451 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.055420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:05.115359 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.115324    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-trusted-ca-bundle\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.115359 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.115360    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0082008-2297-4548-9da2-51af0ceaeec4-console-oauth-config\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.115548 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.115380    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0082008-2297-4548-9da2-51af0ceaeec4-console-serving-cert\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.115548 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.115437    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-console-config\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.115548 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.115463    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-oauth-serving-cert\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.115548 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.115478    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tlrt\" (UniqueName: \"kubernetes.io/projected/d0082008-2297-4548-9da2-51af0ceaeec4-kube-api-access-6tlrt\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.115671 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.115554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-service-ca\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.159229 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.159192    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:05.216968 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.216936    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-console-config\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.217196 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.216978    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-oauth-serving-cert\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.217196 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.216993    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tlrt\" (UniqueName: \"kubernetes.io/projected/d0082008-2297-4548-9da2-51af0ceaeec4-kube-api-access-6tlrt\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.217196 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.217015    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-service-ca\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.217196 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.217054    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-trusted-ca-bundle\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.217196 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.217111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0082008-2297-4548-9da2-51af0ceaeec4-console-oauth-config\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.217196 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.217141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0082008-2297-4548-9da2-51af0ceaeec4-console-serving-cert\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.217769 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.217744    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-console-config\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.218021 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.217957    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-service-ca\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.218144 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.218020    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-oauth-serving-cert\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.218144 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.218129    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0082008-2297-4548-9da2-51af0ceaeec4-trusted-ca-bundle\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.219786 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.219765    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0082008-2297-4548-9da2-51af0ceaeec4-console-serving-cert\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.219838 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.219820    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0082008-2297-4548-9da2-51af0ceaeec4-console-oauth-config\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.228838 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.228819    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tlrt\" (UniqueName: \"kubernetes.io/projected/d0082008-2297-4548-9da2-51af0ceaeec4-kube-api-access-6tlrt\") pod \"console-58cb7f5466-6jqf9\" (UID: \"d0082008-2297-4548-9da2-51af0ceaeec4\") " pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.295894 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.295862    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:05.430153 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:05.430127    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58cb7f5466-6jqf9"]
Jul 16 21:23:05.432429 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:23:05.432399    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0082008_2297_4548_9da2_51af0ceaeec4.slice/crio-56f95b5b5f03fd122a5a212d63d9915b9240722a948cb66665d3f5fef4007e4a WatchSource:0}: Error finding container 56f95b5b5f03fd122a5a212d63d9915b9240722a948cb66665d3f5fef4007e4a: Status 404 returned error can't find the container with id 56f95b5b5f03fd122a5a212d63d9915b9240722a948cb66665d3f5fef4007e4a
Jul 16 21:23:06.265459 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:06.265417    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cb7f5466-6jqf9" event={"ID":"d0082008-2297-4548-9da2-51af0ceaeec4","Type":"ContainerStarted","Data":"0e2006133df40bf51fc45f2c2460801756ecf09b7c7a1720519b2c06456be6f6"}
Jul 16 21:23:06.265916 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:06.265503    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cb7f5466-6jqf9" event={"ID":"d0082008-2297-4548-9da2-51af0ceaeec4","Type":"ContainerStarted","Data":"56f95b5b5f03fd122a5a212d63d9915b9240722a948cb66665d3f5fef4007e4a"}
Jul 16 21:23:06.287467 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:06.287413    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58cb7f5466-6jqf9" podStartSLOduration=2.287398228 podStartE2EDuration="2.287398228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:23:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:23:06.285945072 +0000 UTC m=+450.342775983" watchObservedRunningTime="2026-07-16 21:23:06.287398228 +0000 UTC m=+450.344229137"
Jul 16 21:23:07.295154 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:07.295118    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:11.459256 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:11.459221    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:11.558413 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:11.558377    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:11.863863 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:11.863823    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:11.927125 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:11.927067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:14.258766 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:14.258738    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b9fqb"
Jul 16 21:23:15.296616 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.296579    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:15.297038 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.296645    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:15.301648 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.301619    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:15.462254 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.462224    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:15.562934 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.562854    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:15.940872 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.940792    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"]
Jul 16 21:23:15.944020 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.944005    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:15.945318 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.945296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:15.946241 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.946216    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 16 21:23:15.946241 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.946227    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-smnkp\""
Jul 16 21:23:15.946388 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.946311    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 16 21:23:15.951916 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:15.951892    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"]
Jul 16 21:23:16.015124 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.015065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/827eb8d6-61d7-4815-8054-f8e552de1eae-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vk7hd\" (UID: \"827eb8d6-61d7-4815-8054-f8e552de1eae\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.015318 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.015179    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/827eb8d6-61d7-4815-8054-f8e552de1eae-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk7hd\" (UID: \"827eb8d6-61d7-4815-8054-f8e552de1eae\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.015318 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.015234    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d7s7\" (UniqueName: \"kubernetes.io/projected/827eb8d6-61d7-4815-8054-f8e552de1eae-kube-api-access-2d7s7\") pod \"kuadrant-console-plugin-66b5774985-vk7hd\" (UID: \"827eb8d6-61d7-4815-8054-f8e552de1eae\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.096202 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.096170    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:16.116403 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.116378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/827eb8d6-61d7-4815-8054-f8e552de1eae-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vk7hd\" (UID: \"827eb8d6-61d7-4815-8054-f8e552de1eae\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.116477 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.116441    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/827eb8d6-61d7-4815-8054-f8e552de1eae-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk7hd\" (UID: \"827eb8d6-61d7-4815-8054-f8e552de1eae\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.116517 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.116489    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d7s7\" (UniqueName: \"kubernetes.io/projected/827eb8d6-61d7-4815-8054-f8e552de1eae-kube-api-access-2d7s7\") pod \"kuadrant-console-plugin-66b5774985-vk7hd\" (UID: \"827eb8d6-61d7-4815-8054-f8e552de1eae\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.117056 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.117032    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/827eb8d6-61d7-4815-8054-f8e552de1eae-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vk7hd\" (UID: \"827eb8d6-61d7-4815-8054-f8e552de1eae\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.119106 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.119090    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/827eb8d6-61d7-4815-8054-f8e552de1eae-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk7hd\" (UID: \"827eb8d6-61d7-4815-8054-f8e552de1eae\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.124743 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.124722    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d7s7\" (UniqueName: \"kubernetes.io/projected/827eb8d6-61d7-4815-8054-f8e552de1eae-kube-api-access-2d7s7\") pod \"kuadrant-console-plugin-66b5774985-vk7hd\" (UID: \"827eb8d6-61d7-4815-8054-f8e552de1eae\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.253275 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.253193    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"
Jul 16 21:23:16.322183 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.322152    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58cb7f5466-6jqf9"
Jul 16 21:23:16.391704 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.391675    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd"]
Jul 16 21:23:16.394833 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:23:16.394803    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod827eb8d6_61d7_4815_8054_f8e552de1eae.slice/crio-bda3329ae2e4db5d5998b4e27e72e5b9e39877adf64a9f9cc5c453a422e95161 WatchSource:0}: Error finding container bda3329ae2e4db5d5998b4e27e72e5b9e39877adf64a9f9cc5c453a422e95161: Status 404 returned error can't find the container with id bda3329ae2e4db5d5998b4e27e72e5b9e39877adf64a9f9cc5c453a422e95161
Jul 16 21:23:16.483587 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.483555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:16.539977 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.539945    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:16.618982 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:16.618949    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:17.314677 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:17.314643    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd" event={"ID":"827eb8d6-61d7-4815-8054-f8e552de1eae","Type":"ContainerStarted","Data":"bda3329ae2e4db5d5998b4e27e72e5b9e39877adf64a9f9cc5c453a422e95161"}
Jul 16 21:23:19.081189 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:19.081151    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:19.181751 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:19.181714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:19.283055 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:19.283021    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:19.389367 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:19.389278    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:25.665212 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:25.665157    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:25.767041 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:25.767002    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:25.923895 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:25.923808    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"]
Jul 16 21:23:25.937656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:25.937627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:25.941432 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:25.941401    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zxb6j\""
Jul 16 21:23:25.952559 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:25.952533    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"]
Jul 16 21:23:26.109447 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.109413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5c82\" (UniqueName: \"kubernetes.io/projected/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-kube-api-access-q5c82\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v6xdl\" (UID: \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:26.109622 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.109465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v6xdl\" (UID: \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:26.174658 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.174577    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"]
Jul 16 21:23:26.174900 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:23:26.174875    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-q5c82], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl" podUID="8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca"
Jul 16 21:23:26.209980 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.209949    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v6xdl\" (UID: \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:26.210166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.210112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5c82\" (UniqueName: \"kubernetes.io/projected/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-kube-api-access-q5c82\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v6xdl\" (UID: \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:26.210426 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.210403    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v6xdl\" (UID: \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:26.227967 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.227940    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5c82\" (UniqueName: \"kubernetes.io/projected/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-kube-api-access-q5c82\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v6xdl\" (UID: \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:26.357975 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.357946    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:26.371636 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.371611    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:26.512384 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.512298    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5c82\" (UniqueName: \"kubernetes.io/projected/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-kube-api-access-q5c82\") pod \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\" (UID: \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\") "
Jul 16 21:23:26.512697 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.512669    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-extensions-socket-volume\") pod \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\" (UID: \"8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca\") "
Jul 16 21:23:26.513055 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.513018    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca" (UID: "8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:23:26.513169 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.513125    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-extensions-socket-volume\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:23:26.516498 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.516465    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-kube-api-access-q5c82" (OuterVolumeSpecName: "kube-api-access-q5c82") pod "8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca" (UID: "8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca"). InnerVolumeSpecName "kube-api-access-q5c82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:23:26.536522 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.536489    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"]
Jul 16 21:23:26.546733 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.546700    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"]
Jul 16 21:23:26.614422 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:26.614352    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5c82\" (UniqueName: \"kubernetes.io/projected/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca-kube-api-access-q5c82\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:23:27.363303 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:27.363272    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:27.365697 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:23:27.365658    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-v6xdl\" is forbidden: User \"system:node:ip-10-0-138-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-201.ec2.internal' and this object" podUID="8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:27.373816 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:23:27.373786    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-v6xdl\" is forbidden: User \"system:node:ip-10-0-138-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-201.ec2.internal' and this object" podUID="8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v6xdl"
Jul 16 21:23:28.508128 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:28.508067    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca" path="/var/lib/kubelet/pods/8266d8d1-2ff5-4f8d-a23f-7e76b7c43cca/volumes"
Jul 16 21:23:29.093303 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:29.093270    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:29.193836 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:29.193808    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:31.522587 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:31.522546    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:31.578661 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:31.578624    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:36.881013 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:36.880964    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:36.977341 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:36.977300    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:38.415602 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:38.415556    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd" event={"ID":"827eb8d6-61d7-4815-8054-f8e552de1eae","Type":"ContainerStarted","Data":"d7fef0272a68124ea9dc2c368110743344b563530894b33b1294eaf71796df22"}
Jul 16 21:23:38.439600 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:38.439554    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk7hd" podStartSLOduration=1.8789864490000001 podStartE2EDuration="23.439540198s" totalImagesPullingTime="21.560553749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:23:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:23:38.436760375 +0000 UTC m=+482.493591286" watchObservedRunningTime="2026-07-16 21:23:38.439540198 +0000 UTC m=+482.496371108"
Jul 16 21:23:43.455396 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:43.455331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:43.556103 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:43.556048    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:48.844656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:48.844616    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:48.906759 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:48.906710    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:49.444576 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:49.444539    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:49.499031 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:49.499002    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:53.243345 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:53.243307    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:53.344253 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:53.344215    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:56.854410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:56.854373    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:23:56.954545 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:23:56.954489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:00.037643 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:00.037615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:00.264030 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:00.263996    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:10.704222 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:10.704187    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:10.767666 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:10.767632    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:17.268314 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.268279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:17.312038 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.312004    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plrcn"]
Jul 16 21:24:17.334168 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.334137    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:17.337933 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.337910    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 16 21:24:17.338320 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.338291    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plrcn"]
Jul 16 21:24:17.359674 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.359646    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9p6k\" (UniqueName: \"kubernetes.io/projected/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-kube-api-access-m9p6k\") pod \"limitador-limitador-64c94fd64f-plrcn\" (UID: \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:17.359800 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.359718    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-config-file\") pod \"limitador-limitador-64c94fd64f-plrcn\" (UID: \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:17.408439 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.408398    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:17.410268 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.410242    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plrcn"]
Jul 16 21:24:17.460627 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.460603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9p6k\" (UniqueName: \"kubernetes.io/projected/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-kube-api-access-m9p6k\") pod \"limitador-limitador-64c94fd64f-plrcn\" (UID: \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:17.460768 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.460649    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-config-file\") pod \"limitador-limitador-64c94fd64f-plrcn\" (UID: \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:17.461221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.461202    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-config-file\") pod \"limitador-limitador-64c94fd64f-plrcn\" (UID: \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:17.479123 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.479095    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9p6k\" (UniqueName: \"kubernetes.io/projected/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-kube-api-access-m9p6k\") pod \"limitador-limitador-64c94fd64f-plrcn\" (UID: \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:17.526610 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.526546    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:17.645294 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.645259    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:17.796852 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:17.796700    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plrcn"]
Jul 16 21:24:17.800128 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:24:17.800097    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2056adcd_fdbe_4a0c_8abc_2ebd67f0f958.slice/crio-a31b98177e656e110fcd39d9b85b0eded98deae3137a6e25d1552ef042e95831 WatchSource:0}: Error finding container a31b98177e656e110fcd39d9b85b0eded98deae3137a6e25d1552ef042e95831: Status 404 returned error can't find the container with id a31b98177e656e110fcd39d9b85b0eded98deae3137a6e25d1552ef042e95831
Jul 16 21:24:18.203895 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.203810    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-5wqmh"]
Jul 16 21:24:18.207178 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.207157    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5wqmh"
Jul 16 21:24:18.209716 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:24:18.209689    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"authorino-58cd955675-5wqmh\" is forbidden: User \"system:node:ip-10-0-138-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-201.ec2.internal' and this object" podUID="caf04be2-8294-48a7-821a-552c389e8cf7" pod="kuadrant-system/authorino-58cd955675-5wqmh"
Jul 16 21:24:18.210058 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:24:18.210040    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"authorino-authorino-dockercfg-rp5vz\" is forbidden: User \"system:node:ip-10-0-138-201.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-201.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rp5vz\"" type="*v1.Secret"
Jul 16 21:24:18.224012 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.223991    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5wqmh"]
Jul 16 21:24:18.261049 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.261022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:18.272245 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.267704    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kg76\" (UniqueName: \"kubernetes.io/projected/caf04be2-8294-48a7-821a-552c389e8cf7-kube-api-access-5kg76\") pod \"authorino-58cd955675-5wqmh\" (UID: \"caf04be2-8294-48a7-821a-552c389e8cf7\") " pod="kuadrant-system/authorino-58cd955675-5wqmh"
Jul 16 21:24:18.368167 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.368139    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:18.368306 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.368286    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kg76\" (UniqueName: \"kubernetes.io/projected/caf04be2-8294-48a7-821a-552c389e8cf7-kube-api-access-5kg76\") pod \"authorino-58cd955675-5wqmh\" (UID: \"caf04be2-8294-48a7-821a-552c389e8cf7\") " pod="kuadrant-system/authorino-58cd955675-5wqmh"
Jul 16 21:24:18.387738 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.387710    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kg76\" (UniqueName: \"kubernetes.io/projected/caf04be2-8294-48a7-821a-552c389e8cf7-kube-api-access-5kg76\") pod \"authorino-58cd955675-5wqmh\" (UID: \"caf04be2-8294-48a7-821a-552c389e8cf7\") " pod="kuadrant-system/authorino-58cd955675-5wqmh"
Jul 16 21:24:18.581799 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.581760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn" event={"ID":"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958","Type":"ContainerStarted","Data":"a31b98177e656e110fcd39d9b85b0eded98deae3137a6e25d1552ef042e95831"}
Jul 16 21:24:18.683583 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.683548    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-qxlch"]
Jul 16 21:24:18.687397 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.687353    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qxlch"
Jul 16 21:24:18.700163 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.700134    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qxlch"]
Jul 16 21:24:18.772528 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.772465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6c8n\" (UniqueName: \"kubernetes.io/projected/196dda21-8456-449b-a697-6214ad0bf83f-kube-api-access-w6c8n\") pod \"authorino-75d556bd77-qxlch\" (UID: \"196dda21-8456-449b-a697-6214ad0bf83f\") " pod="kuadrant-system/authorino-75d556bd77-qxlch"
Jul 16 21:24:18.873403 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.873324    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6c8n\" (UniqueName: \"kubernetes.io/projected/196dda21-8456-449b-a697-6214ad0bf83f-kube-api-access-w6c8n\") pod \"authorino-75d556bd77-qxlch\" (UID: \"196dda21-8456-449b-a697-6214ad0bf83f\") " pod="kuadrant-system/authorino-75d556bd77-qxlch"
Jul 16 21:24:18.885839 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:18.885809    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6c8n\" (UniqueName: \"kubernetes.io/projected/196dda21-8456-449b-a697-6214ad0bf83f-kube-api-access-w6c8n\") pod \"authorino-75d556bd77-qxlch\" (UID: \"196dda21-8456-449b-a697-6214ad0bf83f\") " pod="kuadrant-system/authorino-75d556bd77-qxlch"
Jul 16 21:24:19.517259 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:19.517218    2582 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="kuadrant-system/authorino-58cd955675-5wqmh" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Jul 16 21:24:19.517739 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:19.517276    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5wqmh"
Jul 16 21:24:19.670238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:19.670207    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rp5vz\""
Jul 16 21:24:19.672250 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:19.672225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qxlch"
Jul 16 21:24:20.318060 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:20.318026    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qxlch"]
Jul 16 21:24:20.381410 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:24:20.381127    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod196dda21_8456_449b_a697_6214ad0bf83f.slice/crio-2429e9f8e7c6d141cc261e740a1f7627dfea8af30cbe3084bab7d8a21cde6ab6 WatchSource:0}: Error finding container 2429e9f8e7c6d141cc261e740a1f7627dfea8af30cbe3084bab7d8a21cde6ab6: Status 404 returned error can't find the container with id 2429e9f8e7c6d141cc261e740a1f7627dfea8af30cbe3084bab7d8a21cde6ab6
Jul 16 21:24:20.511285 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:20.511251    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5wqmh"]
Jul 16 21:24:20.515197 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:24:20.515173    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcaf04be2_8294_48a7_821a_552c389e8cf7.slice/crio-43738cf78254aeca60f37b748ad5f906e81745f744be5cb72661dc7562bfb499 WatchSource:0}: Error finding container 43738cf78254aeca60f37b748ad5f906e81745f744be5cb72661dc7562bfb499: Status 404 returned error can't find the container with id 43738cf78254aeca60f37b748ad5f906e81745f744be5cb72661dc7562bfb499
Jul 16 21:24:20.592341 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:20.592308    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn" event={"ID":"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958","Type":"ContainerStarted","Data":"894d6220daafa3833debc3337c3e181ea073cf4c314fe8d8faa6d60f68d39903"}
Jul 16 21:24:20.592767 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:20.592457    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:20.593426 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:20.593403    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5wqmh" event={"ID":"caf04be2-8294-48a7-821a-552c389e8cf7","Type":"ContainerStarted","Data":"43738cf78254aeca60f37b748ad5f906e81745f744be5cb72661dc7562bfb499"}
Jul 16 21:24:20.594444 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:20.594426    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qxlch" event={"ID":"196dda21-8456-449b-a697-6214ad0bf83f","Type":"ContainerStarted","Data":"2429e9f8e7c6d141cc261e740a1f7627dfea8af30cbe3084bab7d8a21cde6ab6"}
Jul 16 21:24:20.619810 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:20.619761    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn" podStartSLOduration=0.986206885 podStartE2EDuration="3.619745215s" totalImagesPullingTime="2.63353833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:24:20.616728669 +0000 UTC m=+524.673559580" watchObservedRunningTime="2026-07-16 21:24:20.619745215 +0000 UTC m=+524.676576200"
Jul 16 21:24:24.615438 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:24.615407    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5wqmh" event={"ID":"caf04be2-8294-48a7-821a-552c389e8cf7","Type":"ContainerStarted","Data":"cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1"}
Jul 16 21:24:24.616868 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:24.616843    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qxlch" event={"ID":"196dda21-8456-449b-a697-6214ad0bf83f","Type":"ContainerStarted","Data":"bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b"}
Jul 16 21:24:24.636120 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:24.636051    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-5wqmh" podStartSLOduration=3.358110735 podStartE2EDuration="6.636036719s" totalImagesPullingTime="3.277925984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:24:24.634217835 +0000 UTC m=+528.691048745" watchObservedRunningTime="2026-07-16 21:24:24.636036719 +0000 UTC m=+528.692867629"
Jul 16 21:24:24.656155 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:24.656107    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-qxlch" podStartSLOduration=3.255139814 podStartE2EDuration="6.656059253s" totalImagesPullingTime="3.400919439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:24:24.654247161 +0000 UTC m=+528.711078064" watchObservedRunningTime="2026-07-16 21:24:24.656059253 +0000 UTC m=+528.712890163"
Jul 16 21:24:24.693173 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:24.693139    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5wqmh"]
Jul 16 21:24:26.625699 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:26.625601    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-5wqmh" podUID="caf04be2-8294-48a7-821a-552c389e8cf7" containerName="authorino" containerID="cri-o://cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1" gracePeriod=30
Jul 16 21:24:26.871288 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:26.871262    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5wqmh"
Jul 16 21:24:26.951056 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:26.950921    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kg76\" (UniqueName: \"kubernetes.io/projected/caf04be2-8294-48a7-821a-552c389e8cf7-kube-api-access-5kg76\") pod \"caf04be2-8294-48a7-821a-552c389e8cf7\" (UID: \"caf04be2-8294-48a7-821a-552c389e8cf7\") "
Jul 16 21:24:26.953266 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:26.953201    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf04be2-8294-48a7-821a-552c389e8cf7-kube-api-access-5kg76" (OuterVolumeSpecName: "kube-api-access-5kg76") pod "caf04be2-8294-48a7-821a-552c389e8cf7" (UID: "caf04be2-8294-48a7-821a-552c389e8cf7"). InnerVolumeSpecName "kube-api-access-5kg76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:24:27.051786 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.051741    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kg76\" (UniqueName: \"kubernetes.io/projected/caf04be2-8294-48a7-821a-552c389e8cf7-kube-api-access-5kg76\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:24:27.631256 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.631217    2582 generic.go:356] "Generic (PLEG): container finished" podID="caf04be2-8294-48a7-821a-552c389e8cf7" containerID="cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1" exitCode=0
Jul 16 21:24:27.631256 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.631258    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5wqmh"
Jul 16 21:24:27.631785 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.631278    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5wqmh" event={"ID":"caf04be2-8294-48a7-821a-552c389e8cf7","Type":"ContainerDied","Data":"cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1"}
Jul 16 21:24:27.631785 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.631319    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5wqmh" event={"ID":"caf04be2-8294-48a7-821a-552c389e8cf7","Type":"ContainerDied","Data":"43738cf78254aeca60f37b748ad5f906e81745f744be5cb72661dc7562bfb499"}
Jul 16 21:24:27.631785 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.631337    2582 scope.go:117] "RemoveContainer" containerID="cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1"
Jul 16 21:24:27.640110 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.640070    2582 scope.go:117] "RemoveContainer" containerID="cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1"
Jul 16 21:24:27.640364 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:24:27.640347    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1\": container with ID starting with cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1 not found: ID does not exist" containerID="cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1"
Jul 16 21:24:27.640408 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.640372    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1"} err="failed to get container status \"cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1\": rpc error: code = NotFound desc = could not find container \"cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1\": container with ID starting with cff9550386512f3f28538f578db8fcbb6faa123a29707ab4abcdae452310dfd1 not found: ID does not exist"
Jul 16 21:24:27.655766 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.655740    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5wqmh"]
Jul 16 21:24:27.660509 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:27.660488    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-5wqmh"]
Jul 16 21:24:28.505677 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:28.505645    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caf04be2-8294-48a7-821a-552c389e8cf7" path="/var/lib/kubelet/pods/caf04be2-8294-48a7-821a-552c389e8cf7/volumes"
Jul 16 21:24:31.601415 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:31.601386    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:33.169762 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.169723    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plrcn"]
Jul 16 21:24:33.170166 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.169925    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn" podUID="2056adcd-fdbe-4a0c-8abc-2ebd67f0f958" containerName="limitador" containerID="cri-o://894d6220daafa3833debc3337c3e181ea073cf4c314fe8d8faa6d60f68d39903" gracePeriod=30
Jul 16 21:24:33.658610 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.658575    2582 generic.go:356] "Generic (PLEG): container finished" podID="2056adcd-fdbe-4a0c-8abc-2ebd67f0f958" containerID="894d6220daafa3833debc3337c3e181ea073cf4c314fe8d8faa6d60f68d39903" exitCode=0
Jul 16 21:24:33.658610 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.658602    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn" event={"ID":"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958","Type":"ContainerDied","Data":"894d6220daafa3833debc3337c3e181ea073cf4c314fe8d8faa6d60f68d39903"}
Jul 16 21:24:33.714018 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.713987    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:33.814726 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.814685    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9p6k\" (UniqueName: \"kubernetes.io/projected/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-kube-api-access-m9p6k\") pod \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\" (UID: \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\") "
Jul 16 21:24:33.814903 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.814776    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-config-file\") pod \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\" (UID: \"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958\") "
Jul 16 21:24:33.815158 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.815133    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-config-file" (OuterVolumeSpecName: "config-file") pod "2056adcd-fdbe-4a0c-8abc-2ebd67f0f958" (UID: "2056adcd-fdbe-4a0c-8abc-2ebd67f0f958"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:24:33.816861 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.816838    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-kube-api-access-m9p6k" (OuterVolumeSpecName: "kube-api-access-m9p6k") pod "2056adcd-fdbe-4a0c-8abc-2ebd67f0f958" (UID: "2056adcd-fdbe-4a0c-8abc-2ebd67f0f958"). InnerVolumeSpecName "kube-api-access-m9p6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:24:33.916012 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.915976    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9p6k\" (UniqueName: \"kubernetes.io/projected/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-kube-api-access-m9p6k\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:24:33.916012 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:33.916008    2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958-config-file\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:24:34.663506 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:34.663481    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn"
Jul 16 21:24:34.663506 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:34.663512    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-plrcn" event={"ID":"2056adcd-fdbe-4a0c-8abc-2ebd67f0f958","Type":"ContainerDied","Data":"a31b98177e656e110fcd39d9b85b0eded98deae3137a6e25d1552ef042e95831"}
Jul 16 21:24:34.663954 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:34.663570    2582 scope.go:117] "RemoveContainer" containerID="894d6220daafa3833debc3337c3e181ea073cf4c314fe8d8faa6d60f68d39903"
Jul 16 21:24:34.684710 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:34.684680    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plrcn"]
Jul 16 21:24:34.692694 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:34.692672    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plrcn"]
Jul 16 21:24:36.505759 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:36.505724    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2056adcd-fdbe-4a0c-8abc-2ebd67f0f958" path="/var/lib/kubelet/pods/2056adcd-fdbe-4a0c-8abc-2ebd67f0f958/volumes"
Jul 16 21:24:38.407824 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:38.407786    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:38.503550 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:38.503517    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:38.882419 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:38.882389    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:51.405217 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.405182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:51.673056 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.672978    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dplc"]
Jul 16 21:24:51.673313 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.673301    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf04be2-8294-48a7-821a-552c389e8cf7" containerName="authorino"
Jul 16 21:24:51.673362 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.673314    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf04be2-8294-48a7-821a-552c389e8cf7" containerName="authorino"
Jul 16 21:24:51.673362 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.673328    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2056adcd-fdbe-4a0c-8abc-2ebd67f0f958" containerName="limitador"
Jul 16 21:24:51.673362 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.673333    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2056adcd-fdbe-4a0c-8abc-2ebd67f0f958" containerName="limitador"
Jul 16 21:24:51.673474 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.673388    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2056adcd-fdbe-4a0c-8abc-2ebd67f0f958" containerName="limitador"
Jul 16 21:24:51.673474 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.673398    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf04be2-8294-48a7-821a-552c389e8cf7" containerName="authorino"
Jul 16 21:24:51.680530 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.679679    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7dplc"
Jul 16 21:24:51.710136 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.710104    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dplc"]
Jul 16 21:24:51.736901 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.736867    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:51.775915 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.775885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lchrb\" (UniqueName: \"kubernetes.io/projected/9a881a00-cdbe-41b5-8bd6-3619ba797481-kube-api-access-lchrb\") pod \"authorino-5d4b67c446-7dplc\" (UID: \"9a881a00-cdbe-41b5-8bd6-3619ba797481\") " pod="kuadrant-system/authorino-5d4b67c446-7dplc"
Jul 16 21:24:51.876751 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.876711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lchrb\" (UniqueName: \"kubernetes.io/projected/9a881a00-cdbe-41b5-8bd6-3619ba797481-kube-api-access-lchrb\") pod \"authorino-5d4b67c446-7dplc\" (UID: \"9a881a00-cdbe-41b5-8bd6-3619ba797481\") " pod="kuadrant-system/authorino-5d4b67c446-7dplc"
Jul 16 21:24:51.888825 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.888798    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dplc"]
Jul 16 21:24:51.889051 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:24:51.889035    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lchrb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-7dplc" podUID="9a881a00-cdbe-41b5-8bd6-3619ba797481"
Jul 16 21:24:51.890784 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.890753    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lchrb\" (UniqueName: \"kubernetes.io/projected/9a881a00-cdbe-41b5-8bd6-3619ba797481-kube-api-access-lchrb\") pod \"authorino-5d4b67c446-7dplc\" (UID: \"9a881a00-cdbe-41b5-8bd6-3619ba797481\") " pod="kuadrant-system/authorino-5d4b67c446-7dplc"
Jul 16 21:24:51.917239 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.917206    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84fb57c548-2984w"]
Jul 16 21:24:51.920571 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.920554    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84fb57c548-2984w"
Jul 16 21:24:51.928101 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.928054    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84fb57c548-2984w"]
Jul 16 21:24:51.978044 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:51.978009    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkbbb\" (UniqueName: \"kubernetes.io/projected/a45ac1a2-7c79-479f-af52-e9172c45df1e-kube-api-access-xkbbb\") pod \"authorino-84fb57c548-2984w\" (UID: \"a45ac1a2-7c79-479f-af52-e9172c45df1e\") " pod="kuadrant-system/authorino-84fb57c548-2984w"
Jul 16 21:24:52.078397 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.078363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkbbb\" (UniqueName: \"kubernetes.io/projected/a45ac1a2-7c79-479f-af52-e9172c45df1e-kube-api-access-xkbbb\") pod \"authorino-84fb57c548-2984w\" (UID: \"a45ac1a2-7c79-479f-af52-e9172c45df1e\") " pod="kuadrant-system/authorino-84fb57c548-2984w"
Jul 16 21:24:52.089488 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.089465    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkbbb\" (UniqueName: \"kubernetes.io/projected/a45ac1a2-7c79-479f-af52-e9172c45df1e-kube-api-access-xkbbb\") pod \"authorino-84fb57c548-2984w\" (UID: \"a45ac1a2-7c79-479f-af52-e9172c45df1e\") " pod="kuadrant-system/authorino-84fb57c548-2984w"
Jul 16 21:24:52.105554 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.105521    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84fb57c548-2984w"]
Jul 16 21:24:52.105786 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.105774    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84fb57c548-2984w"
Jul 16 21:24:52.141543 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.141511    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-548f99d44-vdcw4"]
Jul 16 21:24:52.145133 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.145102    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:24:52.148199 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.148178    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 16 21:24:52.161320 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.161271    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-548f99d44-vdcw4"]
Jul 16 21:24:52.178995 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.178929    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-tls-cert\") pod \"authorino-548f99d44-vdcw4\" (UID: \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\") " pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:24:52.179153 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.179038    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfd4z\" (UniqueName: \"kubernetes.io/projected/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-kube-api-access-hfd4z\") pod \"authorino-548f99d44-vdcw4\" (UID: \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\") " pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:24:52.240968 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.240943    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84fb57c548-2984w"]
Jul 16 21:24:52.243652 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:24:52.243613    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda45ac1a2_7c79_479f_af52_e9172c45df1e.slice/crio-1a54e4dbe541f83039b8760c03fb0e21992a534be062a78ce8f66012844cfd2f WatchSource:0}: Error finding container 1a54e4dbe541f83039b8760c03fb0e21992a534be062a78ce8f66012844cfd2f: Status 404 returned error can't find the container with id 1a54e4dbe541f83039b8760c03fb0e21992a534be062a78ce8f66012844cfd2f
Jul 16 21:24:52.279783 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.279750    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfd4z\" (UniqueName: \"kubernetes.io/projected/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-kube-api-access-hfd4z\") pod \"authorino-548f99d44-vdcw4\" (UID: \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\") " pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:24:52.279904 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.279818    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-tls-cert\") pod \"authorino-548f99d44-vdcw4\" (UID: \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\") " pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:24:52.282239 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.282222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-tls-cert\") pod \"authorino-548f99d44-vdcw4\" (UID: \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\") " pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:24:52.288866 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.288847    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfd4z\" (UniqueName: \"kubernetes.io/projected/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-kube-api-access-hfd4z\") pod \"authorino-548f99d44-vdcw4\" (UID: \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\") " pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:24:52.455543 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.455454    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:24:52.582059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.582032    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-548f99d44-vdcw4"]
Jul 16 21:24:52.584140 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:24:52.584118    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7e20aaf_a11f_48b0_a4aa_af1a05d84ee4.slice/crio-435c1333759d8ace1f6edad69888a968c5b33e99a31397396ebb7ac8580d5eec WatchSource:0}: Error finding container 435c1333759d8ace1f6edad69888a968c5b33e99a31397396ebb7ac8580d5eec: Status 404 returned error can't find the container with id 435c1333759d8ace1f6edad69888a968c5b33e99a31397396ebb7ac8580d5eec
Jul 16 21:24:52.740806 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.740700    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548f99d44-vdcw4" event={"ID":"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4","Type":"ContainerStarted","Data":"435c1333759d8ace1f6edad69888a968c5b33e99a31397396ebb7ac8580d5eec"}
Jul 16 21:24:52.743121 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.743048    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84fb57c548-2984w" event={"ID":"a45ac1a2-7c79-479f-af52-e9172c45df1e","Type":"ContainerStarted","Data":"67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869"}
Jul 16 21:24:52.743121 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.743092    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7dplc"
Jul 16 21:24:52.743121 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.743110    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84fb57c548-2984w" event={"ID":"a45ac1a2-7c79-479f-af52-e9172c45df1e","Type":"ContainerStarted","Data":"1a54e4dbe541f83039b8760c03fb0e21992a534be062a78ce8f66012844cfd2f"}
Jul 16 21:24:52.743121 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.743103    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-84fb57c548-2984w" podUID="a45ac1a2-7c79-479f-af52-e9172c45df1e" containerName="authorino" containerID="cri-o://67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869" gracePeriod=30
Jul 16 21:24:52.751421 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.751401    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7dplc"
Jul 16 21:24:52.761637 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.761594    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84fb57c548-2984w" podStartSLOduration=1.3697340709999999 podStartE2EDuration="1.761581152s" totalImagesPullingTime="391.847081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:24:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:24:52.760482623 +0000 UTC m=+556.817313661" watchObservedRunningTime="2026-07-16 21:24:52.761581152 +0000 UTC m=+556.818412041"
Jul 16 21:24:52.784961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.784919    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lchrb\" (UniqueName: \"kubernetes.io/projected/9a881a00-cdbe-41b5-8bd6-3619ba797481-kube-api-access-lchrb\") pod \"9a881a00-cdbe-41b5-8bd6-3619ba797481\" (UID: \"9a881a00-cdbe-41b5-8bd6-3619ba797481\") "
Jul 16 21:24:52.787053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.787030    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a881a00-cdbe-41b5-8bd6-3619ba797481-kube-api-access-lchrb" (OuterVolumeSpecName: "kube-api-access-lchrb") pod "9a881a00-cdbe-41b5-8bd6-3619ba797481" (UID: "9a881a00-cdbe-41b5-8bd6-3619ba797481"). InnerVolumeSpecName "kube-api-access-lchrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:24:52.886666 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.886632    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lchrb\" (UniqueName: \"kubernetes.io/projected/9a881a00-cdbe-41b5-8bd6-3619ba797481-kube-api-access-lchrb\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:24:52.997710 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:52.997653    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84fb57c548-2984w"
Jul 16 21:24:53.088319 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.088296    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkbbb\" (UniqueName: \"kubernetes.io/projected/a45ac1a2-7c79-479f-af52-e9172c45df1e-kube-api-access-xkbbb\") pod \"a45ac1a2-7c79-479f-af52-e9172c45df1e\" (UID: \"a45ac1a2-7c79-479f-af52-e9172c45df1e\") "
Jul 16 21:24:53.090254 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.090224    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a45ac1a2-7c79-479f-af52-e9172c45df1e-kube-api-access-xkbbb" (OuterVolumeSpecName: "kube-api-access-xkbbb") pod "a45ac1a2-7c79-479f-af52-e9172c45df1e" (UID: "a45ac1a2-7c79-479f-af52-e9172c45df1e"). InnerVolumeSpecName "kube-api-access-xkbbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:24:53.188823 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.188788    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkbbb\" (UniqueName: \"kubernetes.io/projected/a45ac1a2-7c79-479f-af52-e9172c45df1e-kube-api-access-xkbbb\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:24:53.748514 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.748480    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548f99d44-vdcw4" event={"ID":"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4","Type":"ContainerStarted","Data":"94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83"}
Jul 16 21:24:53.749715 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.749687    2582 generic.go:356] "Generic (PLEG): container finished" podID="a45ac1a2-7c79-479f-af52-e9172c45df1e" containerID="67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869" exitCode=0
Jul 16 21:24:53.749837 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.749719    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84fb57c548-2984w"
Jul 16 21:24:53.749837 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.749764    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84fb57c548-2984w" event={"ID":"a45ac1a2-7c79-479f-af52-e9172c45df1e","Type":"ContainerDied","Data":"67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869"}
Jul 16 21:24:53.749837 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.749794    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84fb57c548-2984w" event={"ID":"a45ac1a2-7c79-479f-af52-e9172c45df1e","Type":"ContainerDied","Data":"1a54e4dbe541f83039b8760c03fb0e21992a534be062a78ce8f66012844cfd2f"}
Jul 16 21:24:53.749837 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.749810    2582 scope.go:117] "RemoveContainer" containerID="67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869"
Jul 16 21:24:53.750061 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.749923    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7dplc"
Jul 16 21:24:53.759235 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.759215    2582 scope.go:117] "RemoveContainer" containerID="67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869"
Jul 16 21:24:53.759457 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:24:53.759441    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869\": container with ID starting with 67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869 not found: ID does not exist" containerID="67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869"
Jul 16 21:24:53.759502 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.759465    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869"} err="failed to get container status \"67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869\": rpc error: code = NotFound desc = could not find container \"67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869\": container with ID starting with 67095684e6c4dfa522aeb1e2c6fa77959197d25393e15df5a88ee9f26198e869 not found: ID does not exist"
Jul 16 21:24:53.765481 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.765443    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-548f99d44-vdcw4" podStartSLOduration=1.343341929 podStartE2EDuration="1.765429749s" totalImagesPullingTime="422.08782ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:24:53.764813771 +0000 UTC m=+557.821644684" watchObservedRunningTime="2026-07-16 21:24:53.765429749 +0000 UTC m=+557.822260703"
Jul 16 21:24:53.795704 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.795675    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dplc"]
Jul 16 21:24:53.800879 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.800844    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dplc"]
Jul 16 21:24:53.808479 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.808456    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qxlch"]
Jul 16 21:24:53.808669 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.808646    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-qxlch" podUID="196dda21-8456-449b-a697-6214ad0bf83f" containerName="authorino" containerID="cri-o://bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b" gracePeriod=30
Jul 16 21:24:53.812197 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.812172    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84fb57c548-2984w"]
Jul 16 21:24:53.819226 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:53.819206    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84fb57c548-2984w"]
Jul 16 21:24:54.065033 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.065005    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qxlch"
Jul 16 21:24:54.097609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.097584    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6c8n\" (UniqueName: \"kubernetes.io/projected/196dda21-8456-449b-a697-6214ad0bf83f-kube-api-access-w6c8n\") pod \"196dda21-8456-449b-a697-6214ad0bf83f\" (UID: \"196dda21-8456-449b-a697-6214ad0bf83f\") "
Jul 16 21:24:54.099634 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.099611    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/196dda21-8456-449b-a697-6214ad0bf83f-kube-api-access-w6c8n" (OuterVolumeSpecName: "kube-api-access-w6c8n") pod "196dda21-8456-449b-a697-6214ad0bf83f" (UID: "196dda21-8456-449b-a697-6214ad0bf83f"). InnerVolumeSpecName "kube-api-access-w6c8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:24:54.198179 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.198149    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6c8n\" (UniqueName: \"kubernetes.io/projected/196dda21-8456-449b-a697-6214ad0bf83f-kube-api-access-w6c8n\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:24:54.507633 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.507549    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a881a00-cdbe-41b5-8bd6-3619ba797481" path="/var/lib/kubelet/pods/9a881a00-cdbe-41b5-8bd6-3619ba797481/volumes"
Jul 16 21:24:54.507871 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.507852    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a45ac1a2-7c79-479f-af52-e9172c45df1e" path="/var/lib/kubelet/pods/a45ac1a2-7c79-479f-af52-e9172c45df1e/volumes"
Jul 16 21:24:54.756389 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.756355    2582 generic.go:356] "Generic (PLEG): container finished" podID="196dda21-8456-449b-a697-6214ad0bf83f" containerID="bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b" exitCode=0
Jul 16 21:24:54.756782 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.756416    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qxlch"
Jul 16 21:24:54.756782 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.756441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qxlch" event={"ID":"196dda21-8456-449b-a697-6214ad0bf83f","Type":"ContainerDied","Data":"bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b"}
Jul 16 21:24:54.756782 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.756481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qxlch" event={"ID":"196dda21-8456-449b-a697-6214ad0bf83f","Type":"ContainerDied","Data":"2429e9f8e7c6d141cc261e740a1f7627dfea8af30cbe3084bab7d8a21cde6ab6"}
Jul 16 21:24:54.756782 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.756503    2582 scope.go:117] "RemoveContainer" containerID="bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b"
Jul 16 21:24:54.764603 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.764589    2582 scope.go:117] "RemoveContainer" containerID="bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b"
Jul 16 21:24:54.764888 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:24:54.764867    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b\": container with ID starting with bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b not found: ID does not exist" containerID="bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b"
Jul 16 21:24:54.764936 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.764901    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b"} err="failed to get container status \"bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b\": rpc error: code = NotFound desc = could not find container \"bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b\": container with ID starting with bfc9334452b873b2e5e876001e411527edcd19da4c06f4c0f41069a6a257c21b not found: ID does not exist"
Jul 16 21:24:54.774342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.774302    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qxlch"]
Jul 16 21:24:54.777481 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:54.777460    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qxlch"]
Jul 16 21:24:56.506240 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:56.506203    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="196dda21-8456-449b-a697-6214ad0bf83f" path="/var/lib/kubelet/pods/196dda21-8456-449b-a697-6214ad0bf83f/volumes"
Jul 16 21:24:58.207468 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.207433    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:58.457650 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.457574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:58.510977 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.510942    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75646995f5-ztr2d"]
Jul 16 21:24:58.511242 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.511230    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a45ac1a2-7c79-479f-af52-e9172c45df1e" containerName="authorino"
Jul 16 21:24:58.511287 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.511244    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a45ac1a2-7c79-479f-af52-e9172c45df1e" containerName="authorino"
Jul 16 21:24:58.511287 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.511262    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="196dda21-8456-449b-a697-6214ad0bf83f" containerName="authorino"
Jul 16 21:24:58.511287 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.511268    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="196dda21-8456-449b-a697-6214ad0bf83f" containerName="authorino"
Jul 16 21:24:58.511383 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.511312    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a45ac1a2-7c79-479f-af52-e9172c45df1e" containerName="authorino"
Jul 16 21:24:58.511383 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.511323    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="196dda21-8456-449b-a697-6214ad0bf83f" containerName="authorino"
Jul 16 21:24:58.513115 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.513101    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:24:58.515472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.515431    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 21:24:58.515472 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.515464    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mqzw5\""
Jul 16 21:24:58.515642 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.515442    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 21:24:58.527817 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.527790    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75646995f5-ztr2d"]
Jul 16 21:24:58.563700 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.563662    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:58.633766 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.633743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scbzj\" (UniqueName: \"kubernetes.io/projected/e1e254d3-68d8-4d11-a85a-4768d1308db2-kube-api-access-scbzj\") pod \"maas-controller-75646995f5-ztr2d\" (UID: \"e1e254d3-68d8-4d11-a85a-4768d1308db2\") " pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:24:58.633921 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.633789    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1e254d3-68d8-4d11-a85a-4768d1308db2-webhook-cert\") pod \"maas-controller-75646995f5-ztr2d\" (UID: \"e1e254d3-68d8-4d11-a85a-4768d1308db2\") " pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:24:58.634998 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.634977    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:58.729825 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.729757    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:24:58.734726 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.734704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scbzj\" (UniqueName: \"kubernetes.io/projected/e1e254d3-68d8-4d11-a85a-4768d1308db2-kube-api-access-scbzj\") pod \"maas-controller-75646995f5-ztr2d\" (UID: \"e1e254d3-68d8-4d11-a85a-4768d1308db2\") " pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:24:58.734819 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.734756    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1e254d3-68d8-4d11-a85a-4768d1308db2-webhook-cert\") pod \"maas-controller-75646995f5-ztr2d\" (UID: \"e1e254d3-68d8-4d11-a85a-4768d1308db2\") " pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:24:58.737271 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.737253    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1e254d3-68d8-4d11-a85a-4768d1308db2-webhook-cert\") pod \"maas-controller-75646995f5-ztr2d\" (UID: \"e1e254d3-68d8-4d11-a85a-4768d1308db2\") " pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:24:58.745314 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.745294    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scbzj\" (UniqueName: \"kubernetes.io/projected/e1e254d3-68d8-4d11-a85a-4768d1308db2-kube-api-access-scbzj\") pod \"maas-controller-75646995f5-ztr2d\" (UID: \"e1e254d3-68d8-4d11-a85a-4768d1308db2\") " pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:24:58.822950 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.822920    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:24:58.958974 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:58.958946    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75646995f5-ztr2d"]
Jul 16 21:24:58.960605 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:24:58.960583    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e254d3_68d8_4d11_a85a_4768d1308db2.slice/crio-28573458bd5e23b8381a439308d9d1560b5bf0c8619695e24a9610b12f9edb6e WatchSource:0}: Error finding container 28573458bd5e23b8381a439308d9d1560b5bf0c8619695e24a9610b12f9edb6e: Status 404 returned error can't find the container with id 28573458bd5e23b8381a439308d9d1560b5bf0c8619695e24a9610b12f9edb6e
Jul 16 21:24:59.781283 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:24:59.781251    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-ztr2d" event={"ID":"e1e254d3-68d8-4d11-a85a-4768d1308db2","Type":"ContainerStarted","Data":"28573458bd5e23b8381a439308d9d1560b5bf0c8619695e24a9610b12f9edb6e"}
Jul 16 21:25:01.422823 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:01.422793    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:25:01.520409 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:01.520312    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:25:01.793296 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:01.793262    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-ztr2d" event={"ID":"e1e254d3-68d8-4d11-a85a-4768d1308db2","Type":"ContainerStarted","Data":"bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4"}
Jul 16 21:25:01.793500 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:01.793388    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:25:01.824229 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:01.824182    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75646995f5-ztr2d" podStartSLOduration=1.5545004009999999 podStartE2EDuration="3.824166674s" totalImagesPullingTime="2.269666273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:24:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:25:01.823329378 +0000 UTC m=+565.880160289" watchObservedRunningTime="2026-07-16 21:25:01.824166674 +0000 UTC m=+565.880997585"
Jul 16 21:25:12.800194 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:12.800162    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:25:13.212279 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.212188    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75646995f5-ztr2d"]
Jul 16 21:25:13.212452 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.212404    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75646995f5-ztr2d" podUID="e1e254d3-68d8-4d11-a85a-4768d1308db2" containerName="manager" containerID="cri-o://bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4" gracePeriod=30
Jul 16 21:25:13.445197 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.445174    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:25:13.565141 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.565109    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1e254d3-68d8-4d11-a85a-4768d1308db2-webhook-cert\") pod \"e1e254d3-68d8-4d11-a85a-4768d1308db2\" (UID: \"e1e254d3-68d8-4d11-a85a-4768d1308db2\") "
Jul 16 21:25:13.565322 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.565160    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scbzj\" (UniqueName: \"kubernetes.io/projected/e1e254d3-68d8-4d11-a85a-4768d1308db2-kube-api-access-scbzj\") pod \"e1e254d3-68d8-4d11-a85a-4768d1308db2\" (UID: \"e1e254d3-68d8-4d11-a85a-4768d1308db2\") "
Jul 16 21:25:13.567464 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.567429    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e254d3-68d8-4d11-a85a-4768d1308db2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e1e254d3-68d8-4d11-a85a-4768d1308db2" (UID: "e1e254d3-68d8-4d11-a85a-4768d1308db2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:25:13.567464 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.567450    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e254d3-68d8-4d11-a85a-4768d1308db2-kube-api-access-scbzj" (OuterVolumeSpecName: "kube-api-access-scbzj") pod "e1e254d3-68d8-4d11-a85a-4768d1308db2" (UID: "e1e254d3-68d8-4d11-a85a-4768d1308db2"). InnerVolumeSpecName "kube-api-access-scbzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:25:13.665982 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.665940    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1e254d3-68d8-4d11-a85a-4768d1308db2-webhook-cert\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:25:13.665982 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.665974    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scbzj\" (UniqueName: \"kubernetes.io/projected/e1e254d3-68d8-4d11-a85a-4768d1308db2-kube-api-access-scbzj\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:25:13.843926 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.843830    2582 generic.go:356] "Generic (PLEG): container finished" podID="e1e254d3-68d8-4d11-a85a-4768d1308db2" containerID="bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4" exitCode=0
Jul 16 21:25:13.843926 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.843876    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-ztr2d"
Jul 16 21:25:13.844520 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.843917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-ztr2d" event={"ID":"e1e254d3-68d8-4d11-a85a-4768d1308db2","Type":"ContainerDied","Data":"bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4"}
Jul 16 21:25:13.844520 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.843974    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-ztr2d" event={"ID":"e1e254d3-68d8-4d11-a85a-4768d1308db2","Type":"ContainerDied","Data":"28573458bd5e23b8381a439308d9d1560b5bf0c8619695e24a9610b12f9edb6e"}
Jul 16 21:25:13.844520 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.843996    2582 scope.go:117] "RemoveContainer" containerID="bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4"
Jul 16 21:25:13.852428 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.852414    2582 scope.go:117] "RemoveContainer" containerID="bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4"
Jul 16 21:25:13.852666 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:25:13.852648    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4\": container with ID starting with bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4 not found: ID does not exist" containerID="bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4"
Jul 16 21:25:13.852727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.852675    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4"} err="failed to get container status \"bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4\": rpc error: code = NotFound desc = could not find container \"bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4\": container with ID starting with bdfe1069a799c87e1bed6f409191532ec0c2f89b7b7fd7594190b985395b34c4 not found: ID does not exist"
Jul 16 21:25:13.867684 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.867662    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75646995f5-ztr2d"]
Jul 16 21:25:13.872925 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:13.872901    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75646995f5-ztr2d"]
Jul 16 21:25:14.505791 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:14.505747    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1e254d3-68d8-4d11-a85a-4768d1308db2" path="/var/lib/kubelet/pods/e1e254d3-68d8-4d11-a85a-4768d1308db2/volumes"
Jul 16 21:25:33.465626 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:33.465593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:25:34.106528 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.106493    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"]
Jul 16 21:25:34.107637 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.107603    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e254d3-68d8-4d11-a85a-4768d1308db2" containerName="manager"
Jul 16 21:25:34.107637 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.107638    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e254d3-68d8-4d11-a85a-4768d1308db2" containerName="manager"
Jul 16 21:25:34.110029 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.107939    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e254d3-68d8-4d11-a85a-4768d1308db2" containerName="manager"
Jul 16 21:25:34.114262 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.114234    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.117861 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.117845    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 21:25:34.118462 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.118448    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 16 21:25:34.119235 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.119220    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:25:34.140072 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.140042    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"]
Jul 16 21:25:34.145708 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.145683    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e400471d-2484-4358-8038-1ed16965929f-openshift-service-ca\") pod \"maas-api-7954f7d7ff-5mlzf\" (UID: \"e400471d-2484-4358-8038-1ed16965929f\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.145829 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.145727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e400471d-2484-4358-8038-1ed16965929f-maas-api-tls\") pod \"maas-api-7954f7d7ff-5mlzf\" (UID: \"e400471d-2484-4358-8038-1ed16965929f\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.145829 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.145773    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq9x8\" (UniqueName: \"kubernetes.io/projected/e400471d-2484-4358-8038-1ed16965929f-kube-api-access-fq9x8\") pod \"maas-api-7954f7d7ff-5mlzf\" (UID: \"e400471d-2484-4358-8038-1ed16965929f\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.246438 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.246399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e400471d-2484-4358-8038-1ed16965929f-openshift-service-ca\") pod \"maas-api-7954f7d7ff-5mlzf\" (UID: \"e400471d-2484-4358-8038-1ed16965929f\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.246438 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.246445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e400471d-2484-4358-8038-1ed16965929f-maas-api-tls\") pod \"maas-api-7954f7d7ff-5mlzf\" (UID: \"e400471d-2484-4358-8038-1ed16965929f\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.246644 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.246489    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq9x8\" (UniqueName: \"kubernetes.io/projected/e400471d-2484-4358-8038-1ed16965929f-kube-api-access-fq9x8\") pod \"maas-api-7954f7d7ff-5mlzf\" (UID: \"e400471d-2484-4358-8038-1ed16965929f\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.247147 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.247129    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e400471d-2484-4358-8038-1ed16965929f-openshift-service-ca\") pod \"maas-api-7954f7d7ff-5mlzf\" (UID: \"e400471d-2484-4358-8038-1ed16965929f\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.249133 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.249116    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e400471d-2484-4358-8038-1ed16965929f-maas-api-tls\") pod \"maas-api-7954f7d7ff-5mlzf\" (UID: \"e400471d-2484-4358-8038-1ed16965929f\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.258006 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.257973    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq9x8\" (UniqueName: \"kubernetes.io/projected/e400471d-2484-4358-8038-1ed16965929f-kube-api-access-fq9x8\") pod \"maas-api-7954f7d7ff-5mlzf\" (UID: \"e400471d-2484-4358-8038-1ed16965929f\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.423907 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.423819    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:34.558974 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.558948    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"]
Jul 16 21:25:34.562155 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:25:34.562123    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode400471d_2484_4358_8038_1ed16965929f.slice/crio-25633c86982ea23e517b577a4a17b707db791e6dabec033664a00717b48ee275 WatchSource:0}: Error finding container 25633c86982ea23e517b577a4a17b707db791e6dabec033664a00717b48ee275: Status 404 returned error can't find the container with id 25633c86982ea23e517b577a4a17b707db791e6dabec033664a00717b48ee275
Jul 16 21:25:34.704612 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.704530    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"]
Jul 16 21:25:34.707716 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.707696    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:34.709888 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.709867    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qlm45\""
Jul 16 21:25:34.709888 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.709882    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 16 21:25:34.716813 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.716792    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"]
Jul 16 21:25:34.720648 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.720615    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-7zlxf"]
Jul 16 21:25:34.723334 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.723319    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:34.734297 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.734274    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-7zlxf"]
Jul 16 21:25:34.751697 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.751665    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzmpc\" (UniqueName: \"kubernetes.io/projected/596dec7c-a5fb-4aae-8f2d-5903673b4eb4-kube-api-access-bzmpc\") pod \"payload-pre-processing-7677b774bb-gdmsm\" (UID: \"596dec7c-a5fb-4aae-8f2d-5903673b4eb4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:34.751844 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.751709    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596dec7c-a5fb-4aae-8f2d-5903673b4eb4-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-gdmsm\" (UID: \"596dec7c-a5fb-4aae-8f2d-5903673b4eb4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:34.751844 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.751735    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kclmw\" (UniqueName: \"kubernetes.io/projected/42863b88-d149-4131-8598-6921b305389b-kube-api-access-kclmw\") pod \"payload-processing-5bf47cd988-7zlxf\" (UID: \"42863b88-d149-4131-8598-6921b305389b\") " pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:34.751844 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.751819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42863b88-d149-4131-8598-6921b305389b-plugins-config-volume\") pod \"payload-processing-5bf47cd988-7zlxf\" (UID: \"42863b88-d149-4131-8598-6921b305389b\") " pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:34.852951 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.852913    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzmpc\" (UniqueName: \"kubernetes.io/projected/596dec7c-a5fb-4aae-8f2d-5903673b4eb4-kube-api-access-bzmpc\") pod \"payload-pre-processing-7677b774bb-gdmsm\" (UID: \"596dec7c-a5fb-4aae-8f2d-5903673b4eb4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:34.853162 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.852959    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596dec7c-a5fb-4aae-8f2d-5903673b4eb4-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-gdmsm\" (UID: \"596dec7c-a5fb-4aae-8f2d-5903673b4eb4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:34.853162 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.852991    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kclmw\" (UniqueName: \"kubernetes.io/projected/42863b88-d149-4131-8598-6921b305389b-kube-api-access-kclmw\") pod \"payload-processing-5bf47cd988-7zlxf\" (UID: \"42863b88-d149-4131-8598-6921b305389b\") " pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:34.853162 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.853016    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42863b88-d149-4131-8598-6921b305389b-plugins-config-volume\") pod \"payload-processing-5bf47cd988-7zlxf\" (UID: \"42863b88-d149-4131-8598-6921b305389b\") " pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:34.853718 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.853694    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596dec7c-a5fb-4aae-8f2d-5903673b4eb4-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-gdmsm\" (UID: \"596dec7c-a5fb-4aae-8f2d-5903673b4eb4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:34.853818 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.853756    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42863b88-d149-4131-8598-6921b305389b-plugins-config-volume\") pod \"payload-processing-5bf47cd988-7zlxf\" (UID: \"42863b88-d149-4131-8598-6921b305389b\") " pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:34.861813 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.861796    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kclmw\" (UniqueName: \"kubernetes.io/projected/42863b88-d149-4131-8598-6921b305389b-kube-api-access-kclmw\") pod \"payload-processing-5bf47cd988-7zlxf\" (UID: \"42863b88-d149-4131-8598-6921b305389b\") " pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:34.861859 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.861834    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzmpc\" (UniqueName: \"kubernetes.io/projected/596dec7c-a5fb-4aae-8f2d-5903673b4eb4-kube-api-access-bzmpc\") pod \"payload-pre-processing-7677b774bb-gdmsm\" (UID: \"596dec7c-a5fb-4aae-8f2d-5903673b4eb4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:34.933185 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:34.933150    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf" event={"ID":"e400471d-2484-4358-8038-1ed16965929f","Type":"ContainerStarted","Data":"25633c86982ea23e517b577a4a17b707db791e6dabec033664a00717b48ee275"}
Jul 16 21:25:35.019028 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:35.018947    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:35.034150 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:35.034129    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:35.164465 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:35.164226    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"]
Jul 16 21:25:35.167022 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:25:35.166984    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596dec7c_a5fb_4aae_8f2d_5903673b4eb4.slice/crio-d7475a491e5783a360ca4ddc82dabcb26b90ce56fb9d715a97b21b6677353800 WatchSource:0}: Error finding container d7475a491e5783a360ca4ddc82dabcb26b90ce56fb9d715a97b21b6677353800: Status 404 returned error can't find the container with id d7475a491e5783a360ca4ddc82dabcb26b90ce56fb9d715a97b21b6677353800
Jul 16 21:25:35.189700 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:35.189320    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-7zlxf"]
Jul 16 21:25:35.194120 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:25:35.194088    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42863b88_d149_4131_8598_6921b305389b.slice/crio-0eb51431de9ff5ef587deee366fc760faf4f92a4ad7017ac9121f12abf4f4092 WatchSource:0}: Error finding container 0eb51431de9ff5ef587deee366fc760faf4f92a4ad7017ac9121f12abf4f4092: Status 404 returned error can't find the container with id 0eb51431de9ff5ef587deee366fc760faf4f92a4ad7017ac9121f12abf4f4092
Jul 16 21:25:35.939015 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:35.938988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm" event={"ID":"596dec7c-a5fb-4aae-8f2d-5903673b4eb4","Type":"ContainerStarted","Data":"d7475a491e5783a360ca4ddc82dabcb26b90ce56fb9d715a97b21b6677353800"}
Jul 16 21:25:35.941697 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:35.941675    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf" event={"ID":"42863b88-d149-4131-8598-6921b305389b","Type":"ContainerStarted","Data":"0eb51431de9ff5ef587deee366fc760faf4f92a4ad7017ac9121f12abf4f4092"}
Jul 16 21:25:36.948462 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:36.948423    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf" event={"ID":"e400471d-2484-4358-8038-1ed16965929f","Type":"ContainerStarted","Data":"20c299b8d566e665679764f963c51e92f6a78e1dfc1f94702aa60e76b8192858"}
Jul 16 21:25:36.948830 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:36.948526    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:36.975384 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:36.975333    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf" podStartSLOduration=1.653439765 podStartE2EDuration="2.975311954s" totalImagesPullingTime="1.321872189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:25:36.974631206 +0000 UTC m=+601.031462129" watchObservedRunningTime="2026-07-16 21:25:36.975311954 +0000 UTC m=+601.032142865"
Jul 16 21:25:37.954699 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:37.954660    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf" event={"ID":"42863b88-d149-4131-8598-6921b305389b","Type":"ContainerStarted","Data":"819e18abd6a1eff6a6c3da33285aed9a342c9b638664744e6a1e8b8db4639ff6"}
Jul 16 21:25:37.955221 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:37.954843    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:37.956488 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:37.956465    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm" event={"ID":"596dec7c-a5fb-4aae-8f2d-5903673b4eb4","Type":"ContainerStarted","Data":"d900e75080c3272e1bd372224f12a4b7666d1d574647e2f0713e99f551c737d4"}
Jul 16 21:25:37.956607 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:37.956588    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:37.974211 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:37.974170    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf" podStartSLOduration=1.379041671 podStartE2EDuration="3.974155666s" totalImagesPullingTime="2.595113995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:25:37.972803102 +0000 UTC m=+602.029634012" watchObservedRunningTime="2026-07-16 21:25:37.974155666 +0000 UTC m=+602.030986577"
Jul 16 21:25:37.992659 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:37.992606    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm" podStartSLOduration=1.364593613 podStartE2EDuration="3.992591024s" totalImagesPullingTime="2.627997411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:25:37.992238018 +0000 UTC m=+602.049068940" watchObservedRunningTime="2026-07-16 21:25:37.992591024 +0000 UTC m=+602.049421937"
Jul 16 21:25:38.361650 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:38.361604    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:25:42.962039 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:42.962011    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-5mlzf"
Jul 16 21:25:44.950548 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:44.950513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:25:45.047886 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:45.047854    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:25:48.963383 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:48.963354    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-7zlxf"
Jul 16 21:25:48.963758 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:48.963421    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-gdmsm"
Jul 16 21:25:56.794877 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:56.794783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:25:56.897467 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:56.897434    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:25:59.866291 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:25:59.866255    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:26:19.716436 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:19.716394    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:26:19.910530 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:19.910490    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:26:31.067244 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:31.067208    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:26:37.158721 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.158686    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"]
Jul 16 21:26:37.163530 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.163514    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.167959 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.167935    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 21:26:37.168122 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.167989    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5ktjw\""
Jul 16 21:26:37.168122 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.168014    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 21:26:37.168122 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.168022    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 16 21:26:37.177386 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.177352    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"]
Jul 16 21:26:37.181758 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.181734    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:26:37.320512 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.320465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.320512 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.320512    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78197794-2038-40cb-b27e-d68d794efd38-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.320751 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.320559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.320751 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.320594    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v687d\" (UniqueName: \"kubernetes.io/projected/78197794-2038-40cb-b27e-d68d794efd38-kube-api-access-v687d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.320751 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.320638    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.320751 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.320673    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.421381 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.421280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.421381 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.421321    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78197794-2038-40cb-b27e-d68d794efd38-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.421381 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.421341    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.421381 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.421359    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v687d\" (UniqueName: \"kubernetes.io/projected/78197794-2038-40cb-b27e-d68d794efd38-kube-api-access-v687d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.421675 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.421393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.421675 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.421414    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.421760 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.421741    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.421802 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.421779    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.421853 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.421814    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.423778 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.423754    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78197794-2038-40cb-b27e-d68d794efd38-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.423939 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.423923    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78197794-2038-40cb-b27e-d68d794efd38-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.432196 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.432166    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v687d\" (UniqueName: \"kubernetes.io/projected/78197794-2038-40cb-b27e-d68d794efd38-kube-api-access-v687d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kdqqv\" (UID: \"78197794-2038-40cb-b27e-d68d794efd38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.473822 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.473788    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:37.605938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.605913    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"]
Jul 16 21:26:37.608012 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:26:37.607986    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78197794_2038_40cb_b27e_d68d794efd38.slice/crio-1c772979bfab85cf25ee4bbd455d9e471ecbe14e09d189bef583006a4cdff69b WatchSource:0}: Error finding container 1c772979bfab85cf25ee4bbd455d9e471ecbe14e09d189bef583006a4cdff69b: Status 404 returned error can't find the container with id 1c772979bfab85cf25ee4bbd455d9e471ecbe14e09d189bef583006a4cdff69b
Jul 16 21:26:37.609300 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:37.609280    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:26:38.210631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:38.210599    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv" event={"ID":"78197794-2038-40cb-b27e-d68d794efd38","Type":"ContainerStarted","Data":"1c772979bfab85cf25ee4bbd455d9e471ecbe14e09d189bef583006a4cdff69b"}
Jul 16 21:26:39.216200 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:39.216167    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv" event={"ID":"78197794-2038-40cb-b27e-d68d794efd38","Type":"ContainerStarted","Data":"f24f10cb431ddfc68b9f27a215add9705400b17fda47b269bdb17d7df04ac7c6"}
Jul 16 21:26:39.234147 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:39.234098    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv" podStartSLOduration=1.204381526 podStartE2EDuration="2.23406478s" totalImagesPullingTime="1.029683254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:26:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:26:39.233378029 +0000 UTC m=+663.290208941" watchObservedRunningTime="2026-07-16 21:26:39.23406478 +0000 UTC m=+663.290895748"
Jul 16 21:26:47.474446 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:47.474400    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:47.474897 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:47.474465    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:47.487160 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:47.487129    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:48.267048 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:48.267012    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kdqqv"
Jul 16 21:26:54.406461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:54.406430    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:26:57.748890 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.748854    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"]
Jul 16 21:26:57.761400 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.761377    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.764008 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.763983    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 16 21:26:57.765600 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.765576    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"]
Jul 16 21:26:57.792214 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.792186    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:26:57.795389 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.795369    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.795499 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.795396    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.795499 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.795469    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.795568 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.795505    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4add1873-0d5a-4f52-b7ef-6659d151f4eb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.795568 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.795526    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.795630 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.795582    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf6pq\" (UniqueName: \"kubernetes.io/projected/4add1873-0d5a-4f52-b7ef-6659d151f4eb-kube-api-access-kf6pq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.895993 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.895957    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.895993 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.895992    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4add1873-0d5a-4f52-b7ef-6659d151f4eb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.895993 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.896009    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.896343 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.896104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf6pq\" (UniqueName: \"kubernetes.io/projected/4add1873-0d5a-4f52-b7ef-6659d151f4eb-kube-api-access-kf6pq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.896343 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.896160    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.896343 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.896184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.896504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.896470    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.896504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.896486    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.896593 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.896509    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.898563 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.898538    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4add1873-0d5a-4f52-b7ef-6659d151f4eb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.898799 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.898780    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4add1873-0d5a-4f52-b7ef-6659d151f4eb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:57.904286 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:57.904261    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf6pq\" (UniqueName: \"kubernetes.io/projected/4add1873-0d5a-4f52-b7ef-6659d151f4eb-kube-api-access-kf6pq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn\" (UID: \"4add1873-0d5a-4f52-b7ef-6659d151f4eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:58.072703 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:58.072668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:26:58.206010 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:58.205977    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"]
Jul 16 21:26:58.208274 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:26:58.208241    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4add1873_0d5a_4f52_b7ef_6659d151f4eb.slice/crio-66477ba3cd132733b401d4891ba496bb93f3b4753cb89d828bab19a960e537ba WatchSource:0}: Error finding container 66477ba3cd132733b401d4891ba496bb93f3b4753cb89d828bab19a960e537ba: Status 404 returned error can't find the container with id 66477ba3cd132733b401d4891ba496bb93f3b4753cb89d828bab19a960e537ba
Jul 16 21:26:58.300659 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:58.300623    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn" event={"ID":"4add1873-0d5a-4f52-b7ef-6659d151f4eb","Type":"ContainerStarted","Data":"66477ba3cd132733b401d4891ba496bb93f3b4753cb89d828bab19a960e537ba"}
Jul 16 21:26:59.307931 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:59.307888    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn" event={"ID":"4add1873-0d5a-4f52-b7ef-6659d151f4eb","Type":"ContainerStarted","Data":"21d3c0f4bd3f5b5b4ae91c0a28f81578a78aaf96e76185b256233360ab320d32"}
Jul 16 21:26:59.327386 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:26:59.327318    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn" podStartSLOduration=1.987693581 podStartE2EDuration="2.32729353s" totalImagesPullingTime="339.599949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:26:59.326585318 +0000 UTC m=+683.383416228" watchObservedRunningTime="2026-07-16 21:26:59.32729353 +0000 UTC m=+683.384124442"
Jul 16 21:27:08.073312 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:08.073277    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:27:08.073764 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:08.073384    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:27:08.085754 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:08.085732    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:27:08.355889 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:08.355812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn"
Jul 16 21:27:15.988835 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:15.988801    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:27:36.442925 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.442847    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"]
Jul 16 21:27:36.446154 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.446135    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.448924 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.448903    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 16 21:27:36.460714 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.460692    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"]
Jul 16 21:27:36.548781 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.548751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:27:36.549616 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.549593    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.549728 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.549647    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.549728 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.549668    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c56257c8-22a7-4c81-9a13-336e256c6fec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.549843 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.549752    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.549843 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.549804    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5f4c\" (UniqueName: \"kubernetes.io/projected/c56257c8-22a7-4c81-9a13-336e256c6fec-kube-api-access-v5f4c\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.549931 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.549843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.651229 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.651186    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.651435 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.651238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.651435 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.651261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c56257c8-22a7-4c81-9a13-336e256c6fec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.651435 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.651331    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.651435 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.651360    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5f4c\" (UniqueName: \"kubernetes.io/projected/c56257c8-22a7-4c81-9a13-336e256c6fec-kube-api-access-v5f4c\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.651643 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.651526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.651740 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.651634    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.651740 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.651701    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.651902 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.651880    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.653808 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.653784    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c56257c8-22a7-4c81-9a13-336e256c6fec-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.653919 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.653856    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c56257c8-22a7-4c81-9a13-336e256c6fec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.660368 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.660342    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5f4c\" (UniqueName: \"kubernetes.io/projected/c56257c8-22a7-4c81-9a13-336e256c6fec-kube-api-access-v5f4c\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp\" (UID: \"c56257c8-22a7-4c81-9a13-336e256c6fec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.754456 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.754367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:36.887931 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:36.887877    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"]
Jul 16 21:27:37.487495 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:37.487398    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp" event={"ID":"c56257c8-22a7-4c81-9a13-336e256c6fec","Type":"ContainerStarted","Data":"f0ecdfb49e3f3ef959eaa2d6bb14eeb028b68fb21588df68af64501d707adca3"}
Jul 16 21:27:37.487495 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:37.487449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp" event={"ID":"c56257c8-22a7-4c81-9a13-336e256c6fec","Type":"ContainerStarted","Data":"910af52bd64422285a89f198d0b34db40fe21df27cea0c074eeb53aa594a8d4e"}
Jul 16 21:27:37.507159 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:37.507110    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp" podStartSLOduration=1.3252981560000001 podStartE2EDuration="1.507091738s" totalImagesPullingTime="181.793582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:27:37.506134351 +0000 UTC m=+721.562965263" watchObservedRunningTime="2026-07-16 21:27:37.507091738 +0000 UTC m=+721.563922640"
Jul 16 21:27:46.755182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:46.755140    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:46.755182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:46.755192    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:46.767312 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:46.767285    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:27:47.544098 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:27:47.544058    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp"
Jul 16 21:28:29.566342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.566305    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76559f66cc-b9f6c"]
Jul 16 21:28:29.569548 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.569532    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76559f66cc-b9f6c"
Jul 16 21:28:29.578480 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.578454    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76559f66cc-b9f6c"]
Jul 16 21:28:29.648641 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.648605    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5hxg\" (UniqueName: \"kubernetes.io/projected/ce7121eb-d400-4f17-97d6-57bdcea6f7c9-kube-api-access-v5hxg\") pod \"authorino-76559f66cc-b9f6c\" (UID: \"ce7121eb-d400-4f17-97d6-57bdcea6f7c9\") " pod="kuadrant-system/authorino-76559f66cc-b9f6c"
Jul 16 21:28:29.648841 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.648726    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce7121eb-d400-4f17-97d6-57bdcea6f7c9-tls-cert\") pod \"authorino-76559f66cc-b9f6c\" (UID: \"ce7121eb-d400-4f17-97d6-57bdcea6f7c9\") " pod="kuadrant-system/authorino-76559f66cc-b9f6c"
Jul 16 21:28:29.749510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.749473    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hxg\" (UniqueName: \"kubernetes.io/projected/ce7121eb-d400-4f17-97d6-57bdcea6f7c9-kube-api-access-v5hxg\") pod \"authorino-76559f66cc-b9f6c\" (UID: \"ce7121eb-d400-4f17-97d6-57bdcea6f7c9\") " pod="kuadrant-system/authorino-76559f66cc-b9f6c"
Jul 16 21:28:29.749700 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.749569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce7121eb-d400-4f17-97d6-57bdcea6f7c9-tls-cert\") pod \"authorino-76559f66cc-b9f6c\" (UID: \"ce7121eb-d400-4f17-97d6-57bdcea6f7c9\") " pod="kuadrant-system/authorino-76559f66cc-b9f6c"
Jul 16 21:28:29.752097 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.752059    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce7121eb-d400-4f17-97d6-57bdcea6f7c9-tls-cert\") pod \"authorino-76559f66cc-b9f6c\" (UID: \"ce7121eb-d400-4f17-97d6-57bdcea6f7c9\") " pod="kuadrant-system/authorino-76559f66cc-b9f6c"
Jul 16 21:28:29.758812 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.758790    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5hxg\" (UniqueName: \"kubernetes.io/projected/ce7121eb-d400-4f17-97d6-57bdcea6f7c9-kube-api-access-v5hxg\") pod \"authorino-76559f66cc-b9f6c\" (UID: \"ce7121eb-d400-4f17-97d6-57bdcea6f7c9\") " pod="kuadrant-system/authorino-76559f66cc-b9f6c"
Jul 16 21:28:29.880471 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.880382    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76559f66cc-b9f6c"
Jul 16 21:28:29.946301 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:29.946260    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:28:30.014798 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:30.014773    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76559f66cc-b9f6c"]
Jul 16 21:28:30.017104 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:28:30.017059    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce7121eb_d400_4f17_97d6_57bdcea6f7c9.slice/crio-42ab6a4da843c1c0e1046b382169be09940d3462f3ba90ff3c20165688f74c91 WatchSource:0}: Error finding container 42ab6a4da843c1c0e1046b382169be09940d3462f3ba90ff3c20165688f74c91: Status 404 returned error can't find the container with id 42ab6a4da843c1c0e1046b382169be09940d3462f3ba90ff3c20165688f74c91
Jul 16 21:28:30.028380 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:30.028358    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:28:30.736818 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:30.736787    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76559f66cc-b9f6c" event={"ID":"ce7121eb-d400-4f17-97d6-57bdcea6f7c9","Type":"ContainerStarted","Data":"76883209ac2866ab67f5177d956122b84c3f75eab1e4be69cfc4455b49149694"}
Jul 16 21:28:30.737224 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:30.736826    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76559f66cc-b9f6c" event={"ID":"ce7121eb-d400-4f17-97d6-57bdcea6f7c9","Type":"ContainerStarted","Data":"42ab6a4da843c1c0e1046b382169be09940d3462f3ba90ff3c20165688f74c91"}
Jul 16 21:28:30.757184 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:30.757130    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-76559f66cc-b9f6c" podStartSLOduration=1.239879926 podStartE2EDuration="1.757112332s" totalImagesPullingTime="517.232406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:28:30.756016596 +0000 UTC m=+774.812847508" watchObservedRunningTime="2026-07-16 21:28:30.757112332 +0000 UTC m=+774.813943242"
Jul 16 21:28:30.795051 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:30.794957    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-548f99d44-vdcw4"]
Jul 16 21:28:30.795363 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:30.795327    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-548f99d44-vdcw4" podUID="f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4" containerName="authorino" containerID="cri-o://94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83" gracePeriod=30
Jul 16 21:28:31.058674 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.058609    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:28:31.119793 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.119764    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:28:31.163223 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.163184    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-tls-cert\") pod \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\" (UID: \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\") "
Jul 16 21:28:31.163418 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.163258    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfd4z\" (UniqueName: \"kubernetes.io/projected/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-kube-api-access-hfd4z\") pod \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\" (UID: \"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4\") "
Jul 16 21:28:31.165394 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.165361    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-kube-api-access-hfd4z" (OuterVolumeSpecName: "kube-api-access-hfd4z") pod "f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4" (UID: "f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4"). InnerVolumeSpecName "kube-api-access-hfd4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:28:31.173872 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.173839    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4" (UID: "f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:28:31.223062 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.223024    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:28:31.264571 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.264536    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-tls-cert\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:28:31.264571 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.264567    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfd4z\" (UniqueName: \"kubernetes.io/projected/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4-kube-api-access-hfd4z\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:28:31.742704 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.742671    2582 generic.go:356] "Generic (PLEG): container finished" podID="f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4" containerID="94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83" exitCode=0
Jul 16 21:28:31.743175 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.742730    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548f99d44-vdcw4"
Jul 16 21:28:31.743175 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.742760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548f99d44-vdcw4" event={"ID":"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4","Type":"ContainerDied","Data":"94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83"}
Jul 16 21:28:31.743175 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.742801    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548f99d44-vdcw4" event={"ID":"f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4","Type":"ContainerDied","Data":"435c1333759d8ace1f6edad69888a968c5b33e99a31397396ebb7ac8580d5eec"}
Jul 16 21:28:31.743175 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.742819    2582 scope.go:117] "RemoveContainer" containerID="94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83"
Jul 16 21:28:31.756335 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.756297    2582 scope.go:117] "RemoveContainer" containerID="94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83"
Jul 16 21:28:31.756687 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:28:31.756664    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83\": container with ID starting with 94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83 not found: ID does not exist" containerID="94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83"
Jul 16 21:28:31.756787 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.756700    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83"} err="failed to get container status \"94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83\": rpc error: code = NotFound desc = could not find container \"94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83\": container with ID starting with 94cb61049d82c44decfa2258185be0a480ff85f437a1bda8a94a27748245ce83 not found: ID does not exist"
Jul 16 21:28:31.775231 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.775203    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-548f99d44-vdcw4"]
Jul 16 21:28:31.783072 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:31.783045    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-548f99d44-vdcw4"]
Jul 16 21:28:32.509426 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:28:32.509391    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4" path="/var/lib/kubelet/pods/f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4/volumes"
Jul 16 21:29:47.902729 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:29:47.902691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:29:59.231865 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:29:59.231828    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:30:35.429220 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:30:35.429184    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:30:36.676820 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:30:36.676741    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:31:09.200502 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:31:09.200463    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:31:13.630164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:31:13.630135    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:31:13.727765 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:31:13.727726    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:31:13.928007 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:31:13.927920    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:31:20.103589 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:31:20.103554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:31:20.212036 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:31:20.212003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:31:48.636230 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:31:48.636191    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:33:19.242145 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:33:19.242104    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:33:19.346673 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:33:19.346624    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:35:24.678378 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:35:24.678296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:35:24.762070 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:35:24.762017    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:37:58.160258 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:37:58.160161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:38:13.083218 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:38:13.083166    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:38:13.151307 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:38:13.151276    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:40:42.427928 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:40:42.427885    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:41:21.532411 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:41:21.532324    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:41:21.632698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:41:21.632659    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:18.160893 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:18.160856    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:25.563359 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:25.563324    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:33.823328 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:33.823224    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:38.914556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:38.914520    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:41.795100 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:41.795035    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:41.893515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:41.893469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:42.229391 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:42.229308    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:48.079152 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:48.079109    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:48.142355 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:48.142324    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:48.299988 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:48.299956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:48.397680 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:48.397592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:49.504054 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.504018    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:49.602850 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.602818    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"]
Jul 16 21:42:49.603238 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.603224    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4" containerName="authorino"
Jul 16 21:42:49.603297 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.603239    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4" containerName="authorino"
Jul 16 21:42:49.603297 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.603294    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e20aaf-a11f-48b0-a4aa-af1a05d84ee4" containerName="authorino"
Jul 16 21:42:49.606026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.605994    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:49.606189 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.606161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:49.608727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.608709    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fa7d409d-serving-cert\""
Jul 16 21:42:49.608840 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.608749    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-s8nmf\""
Jul 16 21:42:49.619119 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.619069    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"]
Jul 16 21:42:49.646326 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.646290    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjz2\" (UniqueName: \"kubernetes.io/projected/aa98a741-9e5a-44b9-9767-080d6e510a0d-kube-api-access-cdjz2\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:49.646509 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.646342    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa98a741-9e5a-44b9-9767-080d6e510a0d-openshift-service-ca\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:49.646509 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.646379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa98a741-9e5a-44b9-9767-080d6e510a0d-maas-api-tls\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:49.680783 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.680742    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:49.747353 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.747316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjz2\" (UniqueName: \"kubernetes.io/projected/aa98a741-9e5a-44b9-9767-080d6e510a0d-kube-api-access-cdjz2\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:49.747546 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.747373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa98a741-9e5a-44b9-9767-080d6e510a0d-openshift-service-ca\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:49.747546 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.747402    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa98a741-9e5a-44b9-9767-080d6e510a0d-maas-api-tls\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:49.747546 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:42:49.747487    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-serving-cert: secret "maas-api-e2e-ait-fa7d409d-serving-cert" not found
Jul 16 21:42:49.747546 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:42:49.747548    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa98a741-9e5a-44b9-9767-080d6e510a0d-maas-api-tls podName:aa98a741-9e5a-44b9-9767-080d6e510a0d nodeName:}" failed. No retries permitted until 2026-07-16 21:42:50.247532391 +0000 UTC m=+1634.304363280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa98a741-9e5a-44b9-9767-080d6e510a0d-maas-api-tls") pod "maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" (UID: "aa98a741-9e5a-44b9-9767-080d6e510a0d") : secret "maas-api-e2e-ait-fa7d409d-serving-cert" not found
Jul 16 21:42:49.748161 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.748072    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa98a741-9e5a-44b9-9767-080d6e510a0d-openshift-service-ca\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:49.760557 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.760489    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjz2\" (UniqueName: \"kubernetes.io/projected/aa98a741-9e5a-44b9-9767-080d6e510a0d-kube-api-access-cdjz2\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:49.868710 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:49.868669    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:50.252821 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:50.252781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa98a741-9e5a-44b9-9767-080d6e510a0d-maas-api-tls\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:50.255285 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:50.255253    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa98a741-9e5a-44b9-9767-080d6e510a0d-maas-api-tls\") pod \"maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:50.394155 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:50.394123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:50.498922 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:50.498887    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:50.516205 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:50.516120    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:50.654205 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:50.654180    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"]
Jul 16 21:42:50.656114 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:42:50.656047    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa98a741_9e5a_44b9_9767_080d6e510a0d.slice/crio-fbdcbf1248b090ceea76b010b4c7cd6d72e94e31bc971e3647958ac0d4c83a86 WatchSource:0}: Error finding container fbdcbf1248b090ceea76b010b4c7cd6d72e94e31bc971e3647958ac0d4c83a86: Status 404 returned error can't find the container with id fbdcbf1248b090ceea76b010b4c7cd6d72e94e31bc971e3647958ac0d4c83a86
Jul 16 21:42:50.657467 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:50.657445    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:42:50.688779 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:50.688738    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerStarted","Data":"fbdcbf1248b090ceea76b010b4c7cd6d72e94e31bc971e3647958ac0d4c83a86"}
Jul 16 21:42:51.695015 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:51.694979    2582 generic.go:356] "Generic (PLEG): container finished" podID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerID="7dc08d083e292f0e329aa4aa50bb9f4a9b7b4ac90bf016a379fd9dff7252a4d9" exitCode=1
Jul 16 21:42:51.695427 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:51.695055    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerDied","Data":"7dc08d083e292f0e329aa4aa50bb9f4a9b7b4ac90bf016a379fd9dff7252a4d9"}
Jul 16 21:42:51.695427 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:51.695343    2582 scope.go:117] "RemoveContainer" containerID="7dc08d083e292f0e329aa4aa50bb9f4a9b7b4ac90bf016a379fd9dff7252a4d9"
Jul 16 21:42:51.887222 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:51.887170    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:42:52.701594 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:52.701502    2582 generic.go:356] "Generic (PLEG): container finished" podID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerID="15cfd4ecb0ce25376027dd3ff8591bacec100b3e4b193587634ba0b4c191aa7f" exitCode=1
Jul 16 21:42:52.701594 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:52.701569    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerDied","Data":"15cfd4ecb0ce25376027dd3ff8591bacec100b3e4b193587634ba0b4c191aa7f"}
Jul 16 21:42:52.702057 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:52.701630    2582 scope.go:117] "RemoveContainer" containerID="7dc08d083e292f0e329aa4aa50bb9f4a9b7b4ac90bf016a379fd9dff7252a4d9"
Jul 16 21:42:52.702057 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:52.701905    2582 scope.go:117] "RemoveContainer" containerID="15cfd4ecb0ce25376027dd3ff8591bacec100b3e4b193587634ba0b4c191aa7f"
Jul 16 21:42:52.702153 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:42:52.702124    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:42:53.707309 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:53.707282    2582 scope.go:117] "RemoveContainer" containerID="15cfd4ecb0ce25376027dd3ff8591bacec100b3e4b193587634ba0b4c191aa7f"
Jul 16 21:42:53.707711 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:42:53.707456    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:42:55.516859 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:55.516822    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:42:55.517295 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:42:55.517253    2582 scope.go:117] "RemoveContainer" containerID="15cfd4ecb0ce25376027dd3ff8591bacec100b3e4b193587634ba0b4c191aa7f"
Jul 16 21:42:55.517442 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:42:55.517421    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:43:00.516620 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:00.516589    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:43:00.517063 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:00.516922    2582 scope.go:117] "RemoveContainer" containerID="15cfd4ecb0ce25376027dd3ff8591bacec100b3e4b193587634ba0b4c191aa7f"
Jul 16 21:43:00.517134 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:43:00.517101    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:43:14.501426 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:14.501391    2582 scope.go:117] "RemoveContainer" containerID="15cfd4ecb0ce25376027dd3ff8591bacec100b3e4b193587634ba0b4c191aa7f"
Jul 16 21:43:15.809504 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:15.809471    2582 generic.go:356] "Generic (PLEG): container finished" podID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerID="2c8c15fcd8b719ed8cb75ce82e6dddeb4dbbfeb162d724490e58dc97e3ddb795" exitCode=1
Jul 16 21:43:15.809929 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:15.809547    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerDied","Data":"2c8c15fcd8b719ed8cb75ce82e6dddeb4dbbfeb162d724490e58dc97e3ddb795"}
Jul 16 21:43:15.809929 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:15.809606    2582 scope.go:117] "RemoveContainer" containerID="15cfd4ecb0ce25376027dd3ff8591bacec100b3e4b193587634ba0b4c191aa7f"
Jul 16 21:43:15.810068 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:15.809954    2582 scope.go:117] "RemoveContainer" containerID="2c8c15fcd8b719ed8cb75ce82e6dddeb4dbbfeb162d724490e58dc97e3ddb795"
Jul 16 21:43:15.810233 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:43:15.810210    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:43:20.516410 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:20.516378    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:43:20.516802 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:20.516431    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:43:20.516802 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:20.516779    2582 scope.go:117] "RemoveContainer" containerID="2c8c15fcd8b719ed8cb75ce82e6dddeb4dbbfeb162d724490e58dc97e3ddb795"
Jul 16 21:43:20.516992 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:43:20.516972    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:43:32.502006 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:32.501961    2582 scope.go:117] "RemoveContainer" containerID="2c8c15fcd8b719ed8cb75ce82e6dddeb4dbbfeb162d724490e58dc97e3ddb795"
Jul 16 21:43:32.502541 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:43:32.502271    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:43:44.501028 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:44.500993    2582 scope.go:117] "RemoveContainer" containerID="2c8c15fcd8b719ed8cb75ce82e6dddeb4dbbfeb162d724490e58dc97e3ddb795"
Jul 16 21:43:44.941883 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:44.941853    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerStarted","Data":"6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870"}
Jul 16 21:43:44.942098 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:44.942060    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:43:44.968484 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:44.968430    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podStartSLOduration=53.913068782 podStartE2EDuration="55.968413175s" totalImagesPullingTime="2.055344393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:42:49 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:43:44.965628797 +0000 UTC m=+1689.022459715" watchObservedRunningTime="2026-07-16 21:43:44.968413175 +0000 UTC m=+1689.025244105"
Jul 16 21:43:45.947760 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:45.947728    2582 generic.go:356] "Generic (PLEG): container finished" podID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerID="6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870" exitCode=1
Jul 16 21:43:45.948261 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:45.947807    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerDied","Data":"6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870"}
Jul 16 21:43:45.948261 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:45.947860    2582 scope.go:117] "RemoveContainer" containerID="2c8c15fcd8b719ed8cb75ce82e6dddeb4dbbfeb162d724490e58dc97e3ddb795"
Jul 16 21:43:45.948261 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:45.948163    2582 scope.go:117] "RemoveContainer" containerID="6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870"
Jul 16 21:43:45.948416 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:43:45.948390    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:43:46.953540 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:46.953505    2582 scope.go:117] "RemoveContainer" containerID="6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870"
Jul 16 21:43:46.953997 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:43:46.953700    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:43:50.517233 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:50.517202    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:43:50.517637 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:43:50.517584    2582 scope.go:117] "RemoveContainer" containerID="6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870"
Jul 16 21:43:50.517776 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:43:50.517755    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:44:02.501251 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:02.501216    2582 scope.go:117] "RemoveContainer" containerID="6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870"
Jul 16 21:44:02.501635 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:44:02.501413    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:44:15.500833 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:15.500805    2582 scope.go:117] "RemoveContainer" containerID="6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870"
Jul 16 21:44:15.501287 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:44:15.500988    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:44:27.501042 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:27.501005    2582 scope.go:117] "RemoveContainer" containerID="6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870"
Jul 16 21:44:28.135426 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:28.135393    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerStarted","Data":"3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"}
Jul 16 21:44:28.135641 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:28.135624    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:44:29.141249 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:29.141220    2582 generic.go:356] "Generic (PLEG): container finished" podID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba" exitCode=1
Jul 16 21:44:29.141640 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:29.141281    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerDied","Data":"3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"}
Jul 16 21:44:29.141640 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:29.141324    2582 scope.go:117] "RemoveContainer" containerID="6d29b71dd2c01b4437f9e3f8b956cd01018db91b33bd6ac2fd680f806f5f3870"
Jul 16 21:44:29.141640 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:29.141545    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:44:29.141840 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:44:29.141821    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:44:30.147521 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:30.147496    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:44:30.147912 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:44:30.147675    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:44:30.516590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:30.516514    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:44:31.152333 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:31.152306    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:44:31.152740 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:44:31.152496    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:44:44.501746 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:44.501704    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:44:44.502250 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:44:44.501884    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:44:57.500808 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:44:57.500774    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:44:57.501331 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:44:57.500951    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:45:00.175122 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:00.175072    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf"]
Jul 16 21:45:00.178103 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:00.178061    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf"
Jul 16 21:45:00.183508 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:00.183490    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:45:00.187275 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:00.187256    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf"
Jul 16 21:45:00.209647 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:00.209616    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf"]
Jul 16 21:45:00.331019 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:00.330997    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf"]
Jul 16 21:45:00.333123 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:45:00.333096    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1701af_cd02_427d_9f9b_0ec0b6b8b0a1.slice/crio-de2902782a07d8ca4b9248ec8b1bd5de340371f1a80296d8ca80562a16aff78c WatchSource:0}: Error finding container de2902782a07d8ca4b9248ec8b1bd5de340371f1a80296d8ca80562a16aff78c: Status 404 returned error can't find the container with id de2902782a07d8ca4b9248ec8b1bd5de340371f1a80296d8ca80562a16aff78c
Jul 16 21:45:01.287463 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:01.287427    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf" event={"ID":"6c1701af-cd02-427d-9f9b-0ec0b6b8b0a1","Type":"ContainerStarted","Data":"de2902782a07d8ca4b9248ec8b1bd5de340371f1a80296d8ca80562a16aff78c"}
Jul 16 21:45:03.297663 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:03.297626    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf" event={"ID":"6c1701af-cd02-427d-9f9b-0ec0b6b8b0a1","Type":"ContainerStarted","Data":"75aa46133762e7ca0ff12239633fed223f5a5c334c8cb56e370591a9b119f186"}
Jul 16 21:45:03.340934 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:03.340885    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf" podStartSLOduration=1.003857641 podStartE2EDuration="3.34086752s" totalImagesPullingTime="2.337009879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:45:03.335469668 +0000 UTC m=+1767.392300577" watchObservedRunningTime="2026-07-16 21:45:03.34086752 +0000 UTC m=+1767.397698431"
Jul 16 21:45:04.303640 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:04.303608    2582 generic.go:356] "Generic (PLEG): container finished" podID="6c1701af-cd02-427d-9f9b-0ec0b6b8b0a1" containerID="75aa46133762e7ca0ff12239633fed223f5a5c334c8cb56e370591a9b119f186" exitCode=0
Jul 16 21:45:04.304030 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:04.303661    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf" event={"ID":"6c1701af-cd02-427d-9f9b-0ec0b6b8b0a1","Type":"ContainerDied","Data":"75aa46133762e7ca0ff12239633fed223f5a5c334c8cb56e370591a9b119f186"}
Jul 16 21:45:05.439043 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:05.439018    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf"
Jul 16 21:45:06.313146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:06.313108    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf" event={"ID":"6c1701af-cd02-427d-9f9b-0ec0b6b8b0a1","Type":"ContainerDied","Data":"de2902782a07d8ca4b9248ec8b1bd5de340371f1a80296d8ca80562a16aff78c"}
Jul 16 21:45:06.313316 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:06.313166    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de2902782a07d8ca4b9248ec8b1bd5de340371f1a80296d8ca80562a16aff78c"
Jul 16 21:45:06.313316 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:06.313127    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737305-mqbgf"
Jul 16 21:45:12.501641 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:12.501611    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:45:12.502025 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:45:12.501811    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:45:24.501598 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:24.501562    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:45:24.501961 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:45:24.501737    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:45:39.501599 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:39.501566    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:45:39.502035 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:45:39.501767    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:45:51.500872 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:51.500832    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:45:52.530121 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:52.530069    2582 generic.go:356] "Generic (PLEG): container finished" podID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerID="5366c7b6ea6caa3429917ad1f2408619cfefecca694ab79715f4434b6b6f3164" exitCode=1
Jul 16 21:45:52.530121 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:52.530109    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerDied","Data":"5366c7b6ea6caa3429917ad1f2408619cfefecca694ab79715f4434b6b6f3164"}
Jul 16 21:45:52.530581 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:52.530174    2582 scope.go:117] "RemoveContainer" containerID="3ef75a3e0daa0bbea650015ae8981c1af2b9023af034aa0f4a9193d6cfab3dba"
Jul 16 21:45:52.530581 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:52.530511    2582 scope.go:117] "RemoveContainer" containerID="5366c7b6ea6caa3429917ad1f2408619cfefecca694ab79715f4434b6b6f3164"
Jul 16 21:45:52.530735 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:45:52.530715    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:45:53.136005 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:53.135964    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:45:53.537031 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:53.537005    2582 scope.go:117] "RemoveContainer" containerID="5366c7b6ea6caa3429917ad1f2408619cfefecca694ab79715f4434b6b6f3164"
Jul 16 21:45:53.537431 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:45:53.537224    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n_odh-ai-gateway-infra(aa98a741-9e5a-44b9-9767-080d6e510a0d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d"
Jul 16 21:45:54.612061 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:54.612032    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:45:55.839825 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:55.839790    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:45:58.554704 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:58.554651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:45:58.655268 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:45:58.655224    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:00.322869 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.322824    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"]
Jul 16 21:46:00.358715 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.358677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:00.457179 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.457155    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:46:00.574125 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.574016    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"
Jul 16 21:46:00.574125 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.574028    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n" event={"ID":"aa98a741-9e5a-44b9-9767-080d6e510a0d","Type":"ContainerDied","Data":"fbdcbf1248b090ceea76b010b4c7cd6d72e94e31bc971e3647958ac0d4c83a86"}
Jul 16 21:46:00.574125 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.574116    2582 scope.go:117] "RemoveContainer" containerID="5366c7b6ea6caa3429917ad1f2408619cfefecca694ab79715f4434b6b6f3164"
Jul 16 21:46:00.601487 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.601454    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa98a741-9e5a-44b9-9767-080d6e510a0d-openshift-service-ca\") pod \"aa98a741-9e5a-44b9-9767-080d6e510a0d\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") "
Jul 16 21:46:00.601669 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.601499    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdjz2\" (UniqueName: \"kubernetes.io/projected/aa98a741-9e5a-44b9-9767-080d6e510a0d-kube-api-access-cdjz2\") pod \"aa98a741-9e5a-44b9-9767-080d6e510a0d\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") "
Jul 16 21:46:00.601669 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.601559    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa98a741-9e5a-44b9-9767-080d6e510a0d-maas-api-tls\") pod \"aa98a741-9e5a-44b9-9767-080d6e510a0d\" (UID: \"aa98a741-9e5a-44b9-9767-080d6e510a0d\") "
Jul 16 21:46:00.601896 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.601862    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa98a741-9e5a-44b9-9767-080d6e510a0d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa98a741-9e5a-44b9-9767-080d6e510a0d" (UID: "aa98a741-9e5a-44b9-9767-080d6e510a0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:46:00.603977 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.603954    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa98a741-9e5a-44b9-9767-080d6e510a0d-kube-api-access-cdjz2" (OuterVolumeSpecName: "kube-api-access-cdjz2") pod "aa98a741-9e5a-44b9-9767-080d6e510a0d" (UID: "aa98a741-9e5a-44b9-9767-080d6e510a0d"). InnerVolumeSpecName "kube-api-access-cdjz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:46:00.604062 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.603973    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa98a741-9e5a-44b9-9767-080d6e510a0d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa98a741-9e5a-44b9-9767-080d6e510a0d" (UID: "aa98a741-9e5a-44b9-9767-080d6e510a0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:46:00.702820 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.702779    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa98a741-9e5a-44b9-9767-080d6e510a0d-openshift-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.702820 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.702815    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdjz2\" (UniqueName: \"kubernetes.io/projected/aa98a741-9e5a-44b9-9767-080d6e510a0d-kube-api-access-cdjz2\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.702820 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.702829    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa98a741-9e5a-44b9-9767-080d6e510a0d-maas-api-tls\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.896241 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.896199    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"]
Jul 16 21:46:00.900897 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:00.900868    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa7d409d-65d7b894b7-k679n"]
Jul 16 21:46:01.457559 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:01.457504    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:02.507198 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:02.507163    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" path="/var/lib/kubelet/pods/aa98a741-9e5a-44b9-9767-080d6e510a0d/volumes"
Jul 16 21:46:06.765443 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:06.765401    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:07.972740 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:07.972707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:08.076782 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:08.076751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:08.253811 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:08.253713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:08.390295 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:08.390258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:08.513869 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:08.513783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:08.614278 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:08.614246    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:09.843063 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.843033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:09.938271 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938229    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"]
Jul 16 21:46:09.938593 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938579    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938596    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938610    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938616    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938627    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938633    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938642    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c1701af-cd02-427d-9f9b-0ec0b6b8b0a1" containerName="cleanup"
Jul 16 21:46:09.938845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938648    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c1701af-cd02-427d-9f9b-0ec0b6b8b0a1" containerName="cleanup"
Jul 16 21:46:09.938845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938702    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938708    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938714    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938720    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938726    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938733    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:09.938845 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.938740    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c1701af-cd02-427d-9f9b-0ec0b6b8b0a1" containerName="cleanup"
Jul 16 21:46:09.941895 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.941878    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:09.945234 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.945208    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-s8nmf\""
Jul 16 21:46:09.946880 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.946858    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-7c0a35cd-serving-cert\""
Jul 16 21:46:09.953911 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.953885    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"]
Jul 16 21:46:09.976716 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:09.976688    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:10.086253 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.086216    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7h8s\" (UniqueName: \"kubernetes.io/projected/c467d3e0-3d9b-410b-bff7-db97dc156897-kube-api-access-x7h8s\") pod \"maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.086414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.086267    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c467d3e0-3d9b-410b-bff7-db97dc156897-maas-api-tls\") pod \"maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.086414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.086289    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c467d3e0-3d9b-410b-bff7-db97dc156897-openshift-service-ca\") pod \"maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.160830 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.160731    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:10.186991 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.186954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7h8s\" (UniqueName: \"kubernetes.io/projected/c467d3e0-3d9b-410b-bff7-db97dc156897-kube-api-access-x7h8s\") pod \"maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.187186 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.187003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c467d3e0-3d9b-410b-bff7-db97dc156897-maas-api-tls\") pod \"maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.187186 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.187096    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c467d3e0-3d9b-410b-bff7-db97dc156897-openshift-service-ca\") pod \"maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.187791 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.187773    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c467d3e0-3d9b-410b-bff7-db97dc156897-openshift-service-ca\") pod \"maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.189627 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.189606    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c467d3e0-3d9b-410b-bff7-db97dc156897-maas-api-tls\") pod \"maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.197168 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.197143    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7h8s\" (UniqueName: \"kubernetes.io/projected/c467d3e0-3d9b-410b-bff7-db97dc156897-kube-api-access-x7h8s\") pod \"maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.253698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.253667    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:10.403663 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.403634    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"]
Jul 16 21:46:10.406035 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:46:10.406004    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc467d3e0_3d9b_410b_bff7_db97dc156897.slice/crio-fcb9ba20190823b9cb747ba4e3e390de1d92f437147a551578d57dd6e3c3d857 WatchSource:0}: Error finding container fcb9ba20190823b9cb747ba4e3e390de1d92f437147a551578d57dd6e3c3d857: Status 404 returned error can't find the container with id fcb9ba20190823b9cb747ba4e3e390de1d92f437147a551578d57dd6e3c3d857
Jul 16 21:46:10.621959 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.621927    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" event={"ID":"c467d3e0-3d9b-410b-bff7-db97dc156897","Type":"ContainerStarted","Data":"fcb9ba20190823b9cb747ba4e3e390de1d92f437147a551578d57dd6e3c3d857"}
Jul 16 21:46:10.632411 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.632371    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:10.729780 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:10.729752    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:11.628001 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:11.627967    2582 generic.go:356] "Generic (PLEG): container finished" podID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerID="97298d7e5d5d6e865a8c250e482d6d373a62c2d0ca1cd43b59db1a60a9aaabe1" exitCode=1
Jul 16 21:46:11.628485 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:11.628015    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" event={"ID":"c467d3e0-3d9b-410b-bff7-db97dc156897","Type":"ContainerDied","Data":"97298d7e5d5d6e865a8c250e482d6d373a62c2d0ca1cd43b59db1a60a9aaabe1"}
Jul 16 21:46:11.628485 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:11.628346    2582 scope.go:117] "RemoveContainer" containerID="97298d7e5d5d6e865a8c250e482d6d373a62c2d0ca1cd43b59db1a60a9aaabe1"
Jul 16 21:46:11.866674 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:11.866644    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:46:12.634511 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:12.634419    2582 generic.go:356] "Generic (PLEG): container finished" podID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerID="f8550279b22fe426a75907a377ffb908dca76a2c86bd4af50b193a86509f6563" exitCode=1
Jul 16 21:46:12.634511 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:12.634467    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" event={"ID":"c467d3e0-3d9b-410b-bff7-db97dc156897","Type":"ContainerDied","Data":"f8550279b22fe426a75907a377ffb908dca76a2c86bd4af50b193a86509f6563"}
Jul 16 21:46:12.634952 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:12.634517    2582 scope.go:117] "RemoveContainer" containerID="97298d7e5d5d6e865a8c250e482d6d373a62c2d0ca1cd43b59db1a60a9aaabe1"
Jul 16 21:46:12.634952 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:12.634716    2582 scope.go:117] "RemoveContainer" containerID="f8550279b22fe426a75907a377ffb908dca76a2c86bd4af50b193a86509f6563"
Jul 16 21:46:12.635027 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:12.634960    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:46:13.640428 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:13.640397    2582 scope.go:117] "RemoveContainer" containerID="f8550279b22fe426a75907a377ffb908dca76a2c86bd4af50b193a86509f6563"
Jul 16 21:46:13.640832 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:13.640577    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:46:14.245938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.245900    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"]
Jul 16 21:46:14.246290 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.246277    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:14.246351 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.246291    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:14.246351 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.246304    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:14.246351 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.246311    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:14.246351 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.246325    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:14.246351 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.246330    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa98a741-9e5a-44b9-9767-080d6e510a0d" containerName="maas-api"
Jul 16 21:46:14.250564 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.250547    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"
Jul 16 21:46:14.253235 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.253207    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:46:14.256793 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.256766    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"]
Jul 16 21:46:14.321134 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.321061    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd\" (UID: \"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"
Jul 16 21:46:14.421943 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.421898    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd\" (UID: \"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"
Jul 16 21:46:14.422610 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.422588    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd\" (UID: \"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"
Jul 16 21:46:14.563172 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.563134    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"
Jul 16 21:46:14.701040 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:14.701016    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"]
Jul 16 21:46:14.703181 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:46:14.703151    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f9fea8_9ae5_4fdd_bd43_e9768b86f6a3.slice/crio-e82a1abe9bc27a5a47d7850d7da614208463b6d835876a89770c270d989d5c0e WatchSource:0}: Error finding container e82a1abe9bc27a5a47d7850d7da614208463b6d835876a89770c270d989d5c0e: Status 404 returned error can't find the container with id e82a1abe9bc27a5a47d7850d7da614208463b6d835876a89770c270d989d5c0e
Jul 16 21:46:15.254124 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:15.254010    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:15.254455 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:15.254438    2582 scope.go:117] "RemoveContainer" containerID="f8550279b22fe426a75907a377ffb908dca76a2c86bd4af50b193a86509f6563"
Jul 16 21:46:15.254668 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:15.254649    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:46:15.650620 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:15.650586    2582 generic.go:356] "Generic (PLEG): container finished" podID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerID="58bfb1cb045cf65fe65e4b39c761b6461be990819a3e781fb634f24ada8e4358" exitCode=7
Jul 16 21:46:15.650808 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:15.650636    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd" event={"ID":"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3","Type":"ContainerDied","Data":"58bfb1cb045cf65fe65e4b39c761b6461be990819a3e781fb634f24ada8e4358"}
Jul 16 21:46:15.650808 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:15.650693    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd" event={"ID":"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3","Type":"ContainerStarted","Data":"e82a1abe9bc27a5a47d7850d7da614208463b6d835876a89770c270d989d5c0e"}
Jul 16 21:46:15.650940 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:15.650925    2582 scope.go:117] "RemoveContainer" containerID="58bfb1cb045cf65fe65e4b39c761b6461be990819a3e781fb634f24ada8e4358"
Jul 16 21:46:16.657227 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:16.657187    2582 generic.go:356] "Generic (PLEG): container finished" podID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerID="87801617962ed637155c9f2f86f29c94c92041248b6251ab5e4ee2be2847960e" exitCode=7
Jul 16 21:46:16.657656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:16.657275    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd" event={"ID":"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3","Type":"ContainerDied","Data":"87801617962ed637155c9f2f86f29c94c92041248b6251ab5e4ee2be2847960e"}
Jul 16 21:46:16.657656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:16.657324    2582 scope.go:117] "RemoveContainer" containerID="58bfb1cb045cf65fe65e4b39c761b6461be990819a3e781fb634f24ada8e4358"
Jul 16 21:46:16.657656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:16.657527    2582 scope.go:117] "RemoveContainer" containerID="87801617962ed637155c9f2f86f29c94c92041248b6251ab5e4ee2be2847960e"
Jul 16 21:46:16.657779 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:16.657759    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd_odh-ai-gateway-infra(a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3"
Jul 16 21:46:17.664509 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:17.664477    2582 scope.go:117] "RemoveContainer" containerID="87801617962ed637155c9f2f86f29c94c92041248b6251ab5e4ee2be2847960e"
Jul 16 21:46:17.664885 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:17.664770    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd_odh-ai-gateway-infra(a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3"
Jul 16 21:46:20.253814 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:20.253777    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:20.254255 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:20.254241    2582 scope.go:117] "RemoveContainer" containerID="f8550279b22fe426a75907a377ffb908dca76a2c86bd4af50b193a86509f6563"
Jul 16 21:46:20.254467 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:20.254447    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:46:31.501576 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:31.501540    2582 scope.go:117] "RemoveContainer" containerID="87801617962ed637155c9f2f86f29c94c92041248b6251ab5e4ee2be2847960e"
Jul 16 21:46:32.501468 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:32.501383    2582 scope.go:117] "RemoveContainer" containerID="f8550279b22fe426a75907a377ffb908dca76a2c86bd4af50b193a86509f6563"
Jul 16 21:46:32.735533 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:32.735502    2582 generic.go:356] "Generic (PLEG): container finished" podID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerID="74adb5315c0b781f559177ee9d4f26f0020f15ebe7f5dd3ff412700a75bc52d7" exitCode=7
Jul 16 21:46:32.735980 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:32.735565    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd" event={"ID":"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3","Type":"ContainerDied","Data":"74adb5315c0b781f559177ee9d4f26f0020f15ebe7f5dd3ff412700a75bc52d7"}
Jul 16 21:46:32.735980 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:32.735615    2582 scope.go:117] "RemoveContainer" containerID="87801617962ed637155c9f2f86f29c94c92041248b6251ab5e4ee2be2847960e"
Jul 16 21:46:32.735980 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:32.735885    2582 scope.go:117] "RemoveContainer" containerID="74adb5315c0b781f559177ee9d4f26f0020f15ebe7f5dd3ff412700a75bc52d7"
Jul 16 21:46:32.736189 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:32.736140    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd_odh-ai-gateway-infra(a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3"
Jul 16 21:46:33.742346 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:33.742314    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" event={"ID":"c467d3e0-3d9b-410b-bff7-db97dc156897","Type":"ContainerStarted","Data":"ed6840268b09a3ec5c3305c315555e8b7b9e90ca0426e18c4748f7aa63d73706"}
Jul 16 21:46:33.742753 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:33.742607    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:33.767932 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:33.767896    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"]
Jul 16 21:46:33.781576 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:33.781525    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podStartSLOduration=23.056068841 podStartE2EDuration="24.78150827s" totalImagesPullingTime="1.725439429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:46:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:46:33.779918232 +0000 UTC m=+1857.836749147" watchObservedRunningTime="2026-07-16 21:46:33.78150827 +0000 UTC m=+1857.838339182"
Jul 16 21:46:33.901522 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:33.901494    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"
Jul 16 21:46:34.002812 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.002778    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3-maas-api-service-ca\") pod \"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3\" (UID: \"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3\") "
Jul 16 21:46:34.003155 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.003131    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" (UID: "a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:46:34.104263 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.104176    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:46:34.750161 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.750116    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd" event={"ID":"a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3","Type":"ContainerDied","Data":"e82a1abe9bc27a5a47d7850d7da614208463b6d835876a89770c270d989d5c0e"}
Jul 16 21:46:34.750578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.750192    2582 scope.go:117] "RemoveContainer" containerID="74adb5315c0b781f559177ee9d4f26f0020f15ebe7f5dd3ff412700a75bc52d7"
Jul 16 21:46:34.750578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.750137    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"
Jul 16 21:46:34.752303 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.752274    2582 generic.go:356] "Generic (PLEG): container finished" podID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerID="ed6840268b09a3ec5c3305c315555e8b7b9e90ca0426e18c4748f7aa63d73706" exitCode=1
Jul 16 21:46:34.752430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.752325    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" event={"ID":"c467d3e0-3d9b-410b-bff7-db97dc156897","Type":"ContainerDied","Data":"ed6840268b09a3ec5c3305c315555e8b7b9e90ca0426e18c4748f7aa63d73706"}
Jul 16 21:46:34.752656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.752633    2582 scope.go:117] "RemoveContainer" containerID="ed6840268b09a3ec5c3305c315555e8b7b9e90ca0426e18c4748f7aa63d73706"
Jul 16 21:46:34.752923 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:34.752899    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:46:34.759009 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.758989    2582 scope.go:117] "RemoveContainer" containerID="f8550279b22fe426a75907a377ffb908dca76a2c86bd4af50b193a86509f6563"
Jul 16 21:46:34.789941 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.789912    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"]
Jul 16 21:46:34.797031 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:34.797004    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-m6fqd"]
Jul 16 21:46:35.758545 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:35.758511    2582 scope.go:117] "RemoveContainer" containerID="ed6840268b09a3ec5c3305c315555e8b7b9e90ca0426e18c4748f7aa63d73706"
Jul 16 21:46:35.759002 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:35.758740    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:46:36.507028 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:36.507000    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" path="/var/lib/kubelet/pods/a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3/volumes"
Jul 16 21:46:40.254089 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:40.254030    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:46:40.254500 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:40.254453    2582 scope.go:117] "RemoveContainer" containerID="ed6840268b09a3ec5c3305c315555e8b7b9e90ca0426e18c4748f7aa63d73706"
Jul 16 21:46:40.254670 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:40.254647    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:46:52.501407 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:46:52.501374    2582 scope.go:117] "RemoveContainer" containerID="ed6840268b09a3ec5c3305c315555e8b7b9e90ca0426e18c4748f7aa63d73706"
Jul 16 21:46:52.501796 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:46:52.501576    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:47:04.501062 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:04.501025    2582 scope.go:117] "RemoveContainer" containerID="ed6840268b09a3ec5c3305c315555e8b7b9e90ca0426e18c4748f7aa63d73706"
Jul 16 21:47:05.904707 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:05.904672    2582 generic.go:356] "Generic (PLEG): container finished" podID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerID="d2e0de67b83c18b77ab6f3aff248f2c3fc74d517ec3d1b429edfd03098dbe1f0" exitCode=1
Jul 16 21:47:05.905112 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:05.904730    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" event={"ID":"c467d3e0-3d9b-410b-bff7-db97dc156897","Type":"ContainerDied","Data":"d2e0de67b83c18b77ab6f3aff248f2c3fc74d517ec3d1b429edfd03098dbe1f0"}
Jul 16 21:47:05.905112 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:05.904779    2582 scope.go:117] "RemoveContainer" containerID="ed6840268b09a3ec5c3305c315555e8b7b9e90ca0426e18c4748f7aa63d73706"
Jul 16 21:47:05.905112 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:05.905059    2582 scope.go:117] "RemoveContainer" containerID="d2e0de67b83c18b77ab6f3aff248f2c3fc74d517ec3d1b429edfd03098dbe1f0"
Jul 16 21:47:05.905353 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:05.905332    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:47:08.743310 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:08.743261    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:47:08.743693 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:08.743679    2582 scope.go:117] "RemoveContainer" containerID="d2e0de67b83c18b77ab6f3aff248f2c3fc74d517ec3d1b429edfd03098dbe1f0"
Jul 16 21:47:08.743918 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:08.743897    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:47:10.254740 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:10.254699    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:47:10.255183 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:10.255071    2582 scope.go:117] "RemoveContainer" containerID="d2e0de67b83c18b77ab6f3aff248f2c3fc74d517ec3d1b429edfd03098dbe1f0"
Jul 16 21:47:10.255281 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:10.255262    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv_odh-ai-gateway-infra(c467d3e0-3d9b-410b-bff7-db97dc156897)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897"
Jul 16 21:47:14.386443 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:14.386410    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:14.500282 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:14.500250    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:20.016142 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.016043    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"]
Jul 16 21:47:20.144432 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.144407    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:47:20.228809 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.228778    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7h8s\" (UniqueName: \"kubernetes.io/projected/c467d3e0-3d9b-410b-bff7-db97dc156897-kube-api-access-x7h8s\") pod \"c467d3e0-3d9b-410b-bff7-db97dc156897\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") "
Jul 16 21:47:20.228809 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.228816    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c467d3e0-3d9b-410b-bff7-db97dc156897-openshift-service-ca\") pod \"c467d3e0-3d9b-410b-bff7-db97dc156897\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") "
Jul 16 21:47:20.229044 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.228921    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c467d3e0-3d9b-410b-bff7-db97dc156897-maas-api-tls\") pod \"c467d3e0-3d9b-410b-bff7-db97dc156897\" (UID: \"c467d3e0-3d9b-410b-bff7-db97dc156897\") "
Jul 16 21:47:20.229263 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.229238    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c467d3e0-3d9b-410b-bff7-db97dc156897-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c467d3e0-3d9b-410b-bff7-db97dc156897" (UID: "c467d3e0-3d9b-410b-bff7-db97dc156897"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:47:20.231198 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.231165    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c467d3e0-3d9b-410b-bff7-db97dc156897-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c467d3e0-3d9b-410b-bff7-db97dc156897" (UID: "c467d3e0-3d9b-410b-bff7-db97dc156897"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:47:20.231286 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.231173    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c467d3e0-3d9b-410b-bff7-db97dc156897-kube-api-access-x7h8s" (OuterVolumeSpecName: "kube-api-access-x7h8s") pod "c467d3e0-3d9b-410b-bff7-db97dc156897" (UID: "c467d3e0-3d9b-410b-bff7-db97dc156897"). InnerVolumeSpecName "kube-api-access-x7h8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:47:20.330524 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.330489    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c467d3e0-3d9b-410b-bff7-db97dc156897-maas-api-tls\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:47:20.330524 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.330519    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7h8s\" (UniqueName: \"kubernetes.io/projected/c467d3e0-3d9b-410b-bff7-db97dc156897-kube-api-access-x7h8s\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:47:20.330524 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.330529    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c467d3e0-3d9b-410b-bff7-db97dc156897-openshift-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:47:20.468785 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.468747    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:20.970560 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.970522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:20.975759 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.975724    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv" event={"ID":"c467d3e0-3d9b-410b-bff7-db97dc156897","Type":"ContainerDied","Data":"fcb9ba20190823b9cb747ba4e3e390de1d92f437147a551578d57dd6e3c3d857"}
Jul 16 21:47:20.975906 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.975782    2582 scope.go:117] "RemoveContainer" containerID="d2e0de67b83c18b77ab6f3aff248f2c3fc74d517ec3d1b429edfd03098dbe1f0"
Jul 16 21:47:20.975906 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.975836    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"
Jul 16 21:47:20.997958 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:20.997924    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"]
Jul 16 21:47:21.001420 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:21.001392    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7c0a35cd-f9b89757-2mmcv"]
Jul 16 21:47:21.033496 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:21.033463    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:22.507192 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:22.507157    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" path="/var/lib/kubelet/pods/c467d3e0-3d9b-410b-bff7-db97dc156897/volumes"
Jul 16 21:47:26.747928 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:26.747895    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:26.848204 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:26.848163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:26.998465 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:26.998383    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:27.097326 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:27.097294    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:28.185682 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.185649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:28.279177 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.279141    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:28.280165 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280141    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"]
Jul 16 21:47:28.280554 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280537    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:28.280638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280556    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:28.280638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280569    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:28.280638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280577    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:28.280638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280591    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerName="revoke-keys"
Jul 16 21:47:28.280638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280599    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerName="revoke-keys"
Jul 16 21:47:28.280638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280611    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:28.280638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280619    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:28.280638 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280634    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerName="revoke-keys"
Jul 16 21:47:28.281018 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280642    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerName="revoke-keys"
Jul 16 21:47:28.281018 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280740    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:28.281018 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280752    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:28.281018 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280763    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerName="revoke-keys"
Jul 16 21:47:28.281018 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280775    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerName="revoke-keys"
Jul 16 21:47:28.281018 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280784    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerName="revoke-keys"
Jul 16 21:47:28.281018 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.280792    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:28.285798 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.285777    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:28.289005 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.288975    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-s8nmf\""
Jul 16 21:47:28.289132 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.288977    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f6d0b514-serving-cert\""
Jul 16 21:47:28.297821 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.297738    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"]
Jul 16 21:47:28.346602 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.346557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:28.403922 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.403885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fb0a43-a8ca-4838-931f-4e2919bab69e-maas-api-tls\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:28.404131 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.403933    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6sb8\" (UniqueName: \"kubernetes.io/projected/33fb0a43-a8ca-4838-931f-4e2919bab69e-kube-api-access-x6sb8\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:28.404131 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.404045    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fb0a43-a8ca-4838-931f-4e2919bab69e-openshift-service-ca\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:28.505059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.504984    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fb0a43-a8ca-4838-931f-4e2919bab69e-openshift-service-ca\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:28.505059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.505052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fb0a43-a8ca-4838-931f-4e2919bab69e-maas-api-tls\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:28.505247 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.505129    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6sb8\" (UniqueName: \"kubernetes.io/projected/33fb0a43-a8ca-4838-931f-4e2919bab69e-kube-api-access-x6sb8\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:28.505247 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:28.505185    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-serving-cert: secret "maas-api-e2e-ait-f6d0b514-serving-cert" not found
Jul 16 21:47:28.505310 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:28.505264    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33fb0a43-a8ca-4838-931f-4e2919bab69e-maas-api-tls podName:33fb0a43-a8ca-4838-931f-4e2919bab69e nodeName:}" failed. No retries permitted until 2026-07-16 21:47:29.005244481 +0000 UTC m=+1913.062075371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33fb0a43-a8ca-4838-931f-4e2919bab69e-maas-api-tls") pod "maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" (UID: "33fb0a43-a8ca-4838-931f-4e2919bab69e") : secret "maas-api-e2e-ait-f6d0b514-serving-cert" not found
Jul 16 21:47:28.505698 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.505670    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fb0a43-a8ca-4838-931f-4e2919bab69e-openshift-service-ca\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:28.518367 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.518337    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6sb8\" (UniqueName: \"kubernetes.io/projected/33fb0a43-a8ca-4838-931f-4e2919bab69e-kube-api-access-x6sb8\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:28.873466 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.873438    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:28.974917 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:28.974882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:29.010198 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:29.010159    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fb0a43-a8ca-4838-931f-4e2919bab69e-maas-api-tls\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:29.013050 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:29.013023    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fb0a43-a8ca-4838-931f-4e2919bab69e-maas-api-tls\") pod \"maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:29.089327 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:29.089289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:29.197982 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:29.197892    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:29.341629 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:29.341593    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"]
Jul 16 21:47:29.344574 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:47:29.344550    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33fb0a43_a8ca_4838_931f_4e2919bab69e.slice/crio-3e89f1e9f97f75c11d8d64f02a0b6d13ec3c78535fc96400132325d072d0c9aa WatchSource:0}: Error finding container 3e89f1e9f97f75c11d8d64f02a0b6d13ec3c78535fc96400132325d072d0c9aa: Status 404 returned error can't find the container with id 3e89f1e9f97f75c11d8d64f02a0b6d13ec3c78535fc96400132325d072d0c9aa
Jul 16 21:47:30.022944 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:30.022862    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerStarted","Data":"341df63c892bdc4acea3b7219a835b5a0abfdab615ccace7389ed452cc20e6c5"}
Jul 16 21:47:30.022944 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:30.022915    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerStarted","Data":"3e89f1e9f97f75c11d8d64f02a0b6d13ec3c78535fc96400132325d072d0c9aa"}
Jul 16 21:47:30.023178 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:30.022947    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:30.042990 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:30.042742    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podStartSLOduration=1.7129402169999999 podStartE2EDuration="2.042725036s" totalImagesPullingTime="329.784819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:30.039885348 +0000 UTC m=+1914.096716259" watchObservedRunningTime="2026-07-16 21:47:30.042725036 +0000 UTC m=+1914.099555947"
Jul 16 21:47:30.335561 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:30.335528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:31.028508 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:31.028473    2582 generic.go:356] "Generic (PLEG): container finished" podID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerID="341df63c892bdc4acea3b7219a835b5a0abfdab615ccace7389ed452cc20e6c5" exitCode=1
Jul 16 21:47:31.028724 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:31.028561    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerDied","Data":"341df63c892bdc4acea3b7219a835b5a0abfdab615ccace7389ed452cc20e6c5"}
Jul 16 21:47:31.028815 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:31.028801    2582 scope.go:117] "RemoveContainer" containerID="341df63c892bdc4acea3b7219a835b5a0abfdab615ccace7389ed452cc20e6c5"
Jul 16 21:47:32.035567 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:32.035530    2582 generic.go:356] "Generic (PLEG): container finished" podID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerID="b5088ee76df0b35cd35f2d7880b9fa890e90bc79a4a8aa374ca4d9ea25060f92" exitCode=1
Jul 16 21:47:32.035967 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:32.035612    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerDied","Data":"b5088ee76df0b35cd35f2d7880b9fa890e90bc79a4a8aa374ca4d9ea25060f92"}
Jul 16 21:47:32.035967 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:32.035700    2582 scope.go:117] "RemoveContainer" containerID="341df63c892bdc4acea3b7219a835b5a0abfdab615ccace7389ed452cc20e6c5"
Jul 16 21:47:32.036119 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:32.036022    2582 scope.go:117] "RemoveContainer" containerID="b5088ee76df0b35cd35f2d7880b9fa890e90bc79a4a8aa374ca4d9ea25060f92"
Jul 16 21:47:32.036267 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:32.036246    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:47:33.042320 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.042293    2582 scope.go:117] "RemoveContainer" containerID="b5088ee76df0b35cd35f2d7880b9fa890e90bc79a4a8aa374ca4d9ea25060f92"
Jul 16 21:47:33.042694 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:33.042481    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:47:33.192403 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.192359    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"]
Jul 16 21:47:33.192905 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.192887    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerName="revoke-keys"
Jul 16 21:47:33.193021 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.192911    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8f9fea8-9ae5-4fdd-bd43-e9768b86f6a3" containerName="revoke-keys"
Jul 16 21:47:33.193021 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.192953    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:33.193021 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.192962    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:33.193168 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.193053    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c467d3e0-3d9b-410b-bff7-db97dc156897" containerName="maas-api"
Jul 16 21:47:33.196930 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.196914    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"
Jul 16 21:47:33.199409 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.199389    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:47:33.204024 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.203813    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"]
Jul 16 21:47:33.349760 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.349665    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cd57344-76ca-44b7-8037-76e72bf8afb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns\" (UID: \"4cd57344-76ca-44b7-8037-76e72bf8afb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"
Jul 16 21:47:33.450527 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.450486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cd57344-76ca-44b7-8037-76e72bf8afb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns\" (UID: \"4cd57344-76ca-44b7-8037-76e72bf8afb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"
Jul 16 21:47:33.451192 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.451169    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cd57344-76ca-44b7-8037-76e72bf8afb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns\" (UID: \"4cd57344-76ca-44b7-8037-76e72bf8afb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"
Jul 16 21:47:33.508578 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.508544    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"
Jul 16 21:47:33.643780 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:33.643755    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"]
Jul 16 21:47:33.645849 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:47:33.645821    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd57344_76ca_44b7_8037_76e72bf8afb6.slice/crio-ea6fbb60c637aa802222cd6f72057242c6aeeadfa7644be3086b3787f60c45b1 WatchSource:0}: Error finding container ea6fbb60c637aa802222cd6f72057242c6aeeadfa7644be3086b3787f60c45b1: Status 404 returned error can't find the container with id ea6fbb60c637aa802222cd6f72057242c6aeeadfa7644be3086b3787f60c45b1
Jul 16 21:47:34.048149 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.048110    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" event={"ID":"4cd57344-76ca-44b7-8037-76e72bf8afb6","Type":"ContainerStarted","Data":"9f3ea77141890dbf76badcba5ab4ca45c8a605ba18bb8303db50e027d1bbddd0"}
Jul 16 21:47:34.048539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.048168    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" event={"ID":"4cd57344-76ca-44b7-8037-76e72bf8afb6","Type":"ContainerStarted","Data":"ea6fbb60c637aa802222cd6f72057242c6aeeadfa7644be3086b3787f60c45b1"}
Jul 16 21:47:34.066341 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.066292    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" podStartSLOduration=1.066272512 podStartE2EDuration="1.066272512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:34.064070252 +0000 UTC m=+1918.120901193" watchObservedRunningTime="2026-07-16 21:47:34.066272512 +0000 UTC m=+1918.123103424"
Jul 16 21:47:34.521817 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.521725    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:47:34.750648 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.750610    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"]
Jul 16 21:47:34.753960 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.753943    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"
Jul 16 21:47:34.763982 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.763958    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"]
Jul 16 21:47:34.864109 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.864040    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1497c096-c7b6-453d-bba0-d3eeebfd4d5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx\" (UID: \"1497c096-c7b6-453d-bba0-d3eeebfd4d5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"
Jul 16 21:47:34.965434 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.965388    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1497c096-c7b6-453d-bba0-d3eeebfd4d5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx\" (UID: \"1497c096-c7b6-453d-bba0-d3eeebfd4d5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"
Jul 16 21:47:34.966059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:34.966031    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1497c096-c7b6-453d-bba0-d3eeebfd4d5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx\" (UID: \"1497c096-c7b6-453d-bba0-d3eeebfd4d5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"
Jul 16 21:47:35.024028 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:35.023989    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:35.024453 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:35.024439    2582 scope.go:117] "RemoveContainer" containerID="b5088ee76df0b35cd35f2d7880b9fa890e90bc79a4a8aa374ca4d9ea25060f92"
Jul 16 21:47:35.024626 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:35.024607    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:47:35.054585 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:35.054551    2582 generic.go:356] "Generic (PLEG): container finished" podID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerID="9f3ea77141890dbf76badcba5ab4ca45c8a605ba18bb8303db50e027d1bbddd0" exitCode=7
Jul 16 21:47:35.055017 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:35.054634    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" event={"ID":"4cd57344-76ca-44b7-8037-76e72bf8afb6","Type":"ContainerDied","Data":"9f3ea77141890dbf76badcba5ab4ca45c8a605ba18bb8303db50e027d1bbddd0"}
Jul 16 21:47:35.055017 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:35.054829    2582 scope.go:117] "RemoveContainer" containerID="9f3ea77141890dbf76badcba5ab4ca45c8a605ba18bb8303db50e027d1bbddd0"
Jul 16 21:47:35.065745 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:35.065723    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"
Jul 16 21:47:35.220883 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:35.220852    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"]
Jul 16 21:47:35.222786 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:47:35.222466    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1497c096_c7b6_453d_bba0_d3eeebfd4d5f.slice/crio-2f8c76644d1f192ab89c6be2371fac074604320e0ed7f6ac3f52bbb053b3a593 WatchSource:0}: Error finding container 2f8c76644d1f192ab89c6be2371fac074604320e0ed7f6ac3f52bbb053b3a593: Status 404 returned error can't find the container with id 2f8c76644d1f192ab89c6be2371fac074604320e0ed7f6ac3f52bbb053b3a593
Jul 16 21:47:36.061135 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:36.061068    2582 generic.go:356] "Generic (PLEG): container finished" podID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerID="3c690c30aa3ea642aa2d4d4b31ccd38e34251b10656d2ce7928a9862737c2c2f" exitCode=6
Jul 16 21:47:36.061565 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:36.061150    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" event={"ID":"1497c096-c7b6-453d-bba0-d3eeebfd4d5f","Type":"ContainerDied","Data":"3c690c30aa3ea642aa2d4d4b31ccd38e34251b10656d2ce7928a9862737c2c2f"}
Jul 16 21:47:36.061565 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:36.061205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" event={"ID":"1497c096-c7b6-453d-bba0-d3eeebfd4d5f","Type":"ContainerStarted","Data":"2f8c76644d1f192ab89c6be2371fac074604320e0ed7f6ac3f52bbb053b3a593"}
Jul 16 21:47:36.061565 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:36.061408    2582 scope.go:117] "RemoveContainer" containerID="3c690c30aa3ea642aa2d4d4b31ccd38e34251b10656d2ce7928a9862737c2c2f"
Jul 16 21:47:36.063072 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:36.063045    2582 generic.go:356] "Generic (PLEG): container finished" podID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerID="4e2cddd3936ebdf985dfe2b633b4e9178d3928ccdde4e93ea78b60913e2a8b9a" exitCode=7
Jul 16 21:47:36.063198 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:36.063136    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" event={"ID":"4cd57344-76ca-44b7-8037-76e72bf8afb6","Type":"ContainerDied","Data":"4e2cddd3936ebdf985dfe2b633b4e9178d3928ccdde4e93ea78b60913e2a8b9a"}
Jul 16 21:47:36.063198 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:36.063182    2582 scope.go:117] "RemoveContainer" containerID="9f3ea77141890dbf76badcba5ab4ca45c8a605ba18bb8303db50e027d1bbddd0"
Jul 16 21:47:36.063348 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:36.063332    2582 scope.go:117] "RemoveContainer" containerID="4e2cddd3936ebdf985dfe2b633b4e9178d3928ccdde4e93ea78b60913e2a8b9a"
Jul 16 21:47:36.063635 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:36.063542    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns_odh-ai-gateway-infra(4cd57344-76ca-44b7-8037-76e72bf8afb6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6"
Jul 16 21:47:37.069627 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:37.069601    2582 scope.go:117] "RemoveContainer" containerID="4e2cddd3936ebdf985dfe2b633b4e9178d3928ccdde4e93ea78b60913e2a8b9a"
Jul 16 21:47:37.070118 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:37.069841    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns_odh-ai-gateway-infra(4cd57344-76ca-44b7-8037-76e72bf8afb6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6"
Jul 16 21:47:37.071180 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:37.071148    2582 generic.go:356] "Generic (PLEG): container finished" podID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerID="18bd947440322cd8ee13dd87aa1ae0298e04942e6aae535f5cdbe64fccf1557a" exitCode=6
Jul 16 21:47:37.071326 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:37.071216    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" event={"ID":"1497c096-c7b6-453d-bba0-d3eeebfd4d5f","Type":"ContainerDied","Data":"18bd947440322cd8ee13dd87aa1ae0298e04942e6aae535f5cdbe64fccf1557a"}
Jul 16 21:47:37.071326 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:37.071271    2582 scope.go:117] "RemoveContainer" containerID="3c690c30aa3ea642aa2d4d4b31ccd38e34251b10656d2ce7928a9862737c2c2f"
Jul 16 21:47:37.071416 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:37.071383    2582 scope.go:117] "RemoveContainer" containerID="18bd947440322cd8ee13dd87aa1ae0298e04942e6aae535f5cdbe64fccf1557a"
Jul 16 21:47:37.071607 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:37.071589    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx_odh-ai-gateway-infra(1497c096-c7b6-453d-bba0-d3eeebfd4d5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f"
Jul 16 21:47:38.079037 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:38.079001    2582 scope.go:117] "RemoveContainer" containerID="18bd947440322cd8ee13dd87aa1ae0298e04942e6aae535f5cdbe64fccf1557a"
Jul 16 21:47:38.079479 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:38.079223    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx_odh-ai-gateway-infra(1497c096-c7b6-453d-bba0-d3eeebfd4d5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f"
Jul 16 21:47:39.198440 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:39.198352    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:39.198788 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:39.198757    2582 scope.go:117] "RemoveContainer" containerID="b5088ee76df0b35cd35f2d7880b9fa890e90bc79a4a8aa374ca4d9ea25060f92"
Jul 16 21:47:39.198953 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:39.198932    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:47:49.501378 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:49.501344    2582 scope.go:117] "RemoveContainer" containerID="4e2cddd3936ebdf985dfe2b633b4e9178d3928ccdde4e93ea78b60913e2a8b9a"
Jul 16 21:47:50.139230 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:50.139142    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" event={"ID":"4cd57344-76ca-44b7-8037-76e72bf8afb6","Type":"ContainerStarted","Data":"31c363dce4f094ea2e1630422e6cebbed1ea8b0380896a4ec3362b5986cb4102"}
Jul 16 21:47:51.145481 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:51.145447    2582 generic.go:356] "Generic (PLEG): container finished" podID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerID="31c363dce4f094ea2e1630422e6cebbed1ea8b0380896a4ec3362b5986cb4102" exitCode=7
Jul 16 21:47:51.145925 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:51.145507    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" event={"ID":"4cd57344-76ca-44b7-8037-76e72bf8afb6","Type":"ContainerDied","Data":"31c363dce4f094ea2e1630422e6cebbed1ea8b0380896a4ec3362b5986cb4102"}
Jul 16 21:47:51.145925 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:51.145558    2582 scope.go:117] "RemoveContainer" containerID="4e2cddd3936ebdf985dfe2b633b4e9178d3928ccdde4e93ea78b60913e2a8b9a"
Jul 16 21:47:51.145925 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:51.145831    2582 scope.go:117] "RemoveContainer" containerID="31c363dce4f094ea2e1630422e6cebbed1ea8b0380896a4ec3362b5986cb4102"
Jul 16 21:47:51.146109 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:51.146043    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns_odh-ai-gateway-infra(4cd57344-76ca-44b7-8037-76e72bf8afb6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6"
Jul 16 21:47:51.168176 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:51.168144    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"]
Jul 16 21:47:51.500882 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:51.500795    2582 scope.go:117] "RemoveContainer" containerID="b5088ee76df0b35cd35f2d7880b9fa890e90bc79a4a8aa374ca4d9ea25060f92"
Jul 16 21:47:51.501130 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:51.501117    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:47:52.279507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:52.279480    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"
Jul 16 21:47:52.308911 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:52.308885    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cd57344-76ca-44b7-8037-76e72bf8afb6-maas-api-service-ca\") pod \"4cd57344-76ca-44b7-8037-76e72bf8afb6\" (UID: \"4cd57344-76ca-44b7-8037-76e72bf8afb6\") "
Jul 16 21:47:52.309243 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:52.309222    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cd57344-76ca-44b7-8037-76e72bf8afb6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4cd57344-76ca-44b7-8037-76e72bf8afb6" (UID: "4cd57344-76ca-44b7-8037-76e72bf8afb6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:47:52.410646 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:52.410556    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cd57344-76ca-44b7-8037-76e72bf8afb6-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:47:52.501725 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:52.501696    2582 scope.go:117] "RemoveContainer" containerID="18bd947440322cd8ee13dd87aa1ae0298e04942e6aae535f5cdbe64fccf1557a"
Jul 16 21:47:53.160054 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.160021    2582 generic.go:356] "Generic (PLEG): container finished" podID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerID="3a107e9a858fa873350acea230a86f863dda95b6803aa25b38b5d70f84c34ae9" exitCode=1
Jul 16 21:47:53.160284 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.160105    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerDied","Data":"3a107e9a858fa873350acea230a86f863dda95b6803aa25b38b5d70f84c34ae9"}
Jul 16 21:47:53.160284 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.160168    2582 scope.go:117] "RemoveContainer" containerID="b5088ee76df0b35cd35f2d7880b9fa890e90bc79a4a8aa374ca4d9ea25060f92"
Jul 16 21:47:53.160515 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.160493    2582 scope.go:117] "RemoveContainer" containerID="3a107e9a858fa873350acea230a86f863dda95b6803aa25b38b5d70f84c34ae9"
Jul 16 21:47:53.160749 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:53.160729    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:47:53.162005 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.161975    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" event={"ID":"1497c096-c7b6-453d-bba0-d3eeebfd4d5f","Type":"ContainerStarted","Data":"10b641bf6fb86f0c75b5d90b44aef9d2f08fed82fdd1d703ea4c567b6d56756f"}
Jul 16 21:47:53.163232 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.163212    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns" event={"ID":"4cd57344-76ca-44b7-8037-76e72bf8afb6","Type":"ContainerDied","Data":"ea6fbb60c637aa802222cd6f72057242c6aeeadfa7644be3086b3787f60c45b1"}
Jul 16 21:47:53.163232 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.163225    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"
Jul 16 21:47:53.171231 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.171211    2582 scope.go:117] "RemoveContainer" containerID="31c363dce4f094ea2e1630422e6cebbed1ea8b0380896a4ec3362b5986cb4102"
Jul 16 21:47:53.192484 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.192456    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"]
Jul 16 21:47:53.198126 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.198093    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-wg8ns"]
Jul 16 21:47:53.212493 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:53.212449    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" podStartSLOduration=19.212435438 podStartE2EDuration="19.212435438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:53.210745136 +0000 UTC m=+1937.267576048" watchObservedRunningTime="2026-07-16 21:47:53.212435438 +0000 UTC m=+1937.269266350"
Jul 16 21:47:54.171909 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:54.171872    2582 generic.go:356] "Generic (PLEG): container finished" podID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerID="10b641bf6fb86f0c75b5d90b44aef9d2f08fed82fdd1d703ea4c567b6d56756f" exitCode=6
Jul 16 21:47:54.172412 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:54.171948    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" event={"ID":"1497c096-c7b6-453d-bba0-d3eeebfd4d5f","Type":"ContainerDied","Data":"10b641bf6fb86f0c75b5d90b44aef9d2f08fed82fdd1d703ea4c567b6d56756f"}
Jul 16 21:47:54.172412 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:54.172019    2582 scope.go:117] "RemoveContainer" containerID="18bd947440322cd8ee13dd87aa1ae0298e04942e6aae535f5cdbe64fccf1557a"
Jul 16 21:47:54.172412 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:54.172332    2582 scope.go:117] "RemoveContainer" containerID="10b641bf6fb86f0c75b5d90b44aef9d2f08fed82fdd1d703ea4c567b6d56756f"
Jul 16 21:47:54.172586 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:54.172565    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx_odh-ai-gateway-infra(1497c096-c7b6-453d-bba0-d3eeebfd4d5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f"
Jul 16 21:47:54.222382 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:54.222347    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"]
Jul 16 21:47:54.506993 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:54.506909    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" path="/var/lib/kubelet/pods/4cd57344-76ca-44b7-8037-76e72bf8afb6/volumes"
Jul 16 21:47:55.023168 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:55.023137    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:55.023497 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:55.023484    2582 scope.go:117] "RemoveContainer" containerID="3a107e9a858fa873350acea230a86f863dda95b6803aa25b38b5d70f84c34ae9"
Jul 16 21:47:55.023677 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:55.023659    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:47:55.316137 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:55.316113    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"
Jul 16 21:47:55.340757 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:55.340728    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1497c096-c7b6-453d-bba0-d3eeebfd4d5f-maas-api-service-ca\") pod \"1497c096-c7b6-453d-bba0-d3eeebfd4d5f\" (UID: \"1497c096-c7b6-453d-bba0-d3eeebfd4d5f\") "
Jul 16 21:47:55.341139 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:55.341117    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1497c096-c7b6-453d-bba0-d3eeebfd4d5f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1497c096-c7b6-453d-bba0-d3eeebfd4d5f" (UID: "1497c096-c7b6-453d-bba0-d3eeebfd4d5f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:47:55.441757 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:55.441724    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1497c096-c7b6-453d-bba0-d3eeebfd4d5f-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:47:56.185832 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:56.185807    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"
Jul 16 21:47:56.185989 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:56.185804    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx" event={"ID":"1497c096-c7b6-453d-bba0-d3eeebfd4d5f","Type":"ContainerDied","Data":"2f8c76644d1f192ab89c6be2371fac074604320e0ed7f6ac3f52bbb053b3a593"}
Jul 16 21:47:56.185989 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:56.185922    2582 scope.go:117] "RemoveContainer" containerID="10b641bf6fb86f0c75b5d90b44aef9d2f08fed82fdd1d703ea4c567b6d56756f"
Jul 16 21:47:56.208168 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:56.208137    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"]
Jul 16 21:47:56.214339 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:56.214300    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-8rvpx"]
Jul 16 21:47:56.507210 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:56.507125    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" path="/var/lib/kubelet/pods/1497c096-c7b6-453d-bba0-d3eeebfd4d5f/volumes"
Jul 16 21:47:59.198004 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:59.197963    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:47:59.198411 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:47:59.198367    2582 scope.go:117] "RemoveContainer" containerID="3a107e9a858fa873350acea230a86f863dda95b6803aa25b38b5d70f84c34ae9"
Jul 16 21:47:59.198562 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:47:59.198542    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:48:03.914858 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:03.914825    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:48:12.501142 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:12.501108    2582 scope.go:117] "RemoveContainer" containerID="3a107e9a858fa873350acea230a86f863dda95b6803aa25b38b5d70f84c34ae9"
Jul 16 21:48:12.501544 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:48:12.501294    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:48:24.501766 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:24.501733    2582 scope.go:117] "RemoveContainer" containerID="3a107e9a858fa873350acea230a86f863dda95b6803aa25b38b5d70f84c34ae9"
Jul 16 21:48:25.318831 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:25.318792    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerStarted","Data":"13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79"}
Jul 16 21:48:25.319051 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:25.319034    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:48:26.334314 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:26.334267    2582 generic.go:356] "Generic (PLEG): container finished" podID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerID="13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79" exitCode=1
Jul 16 21:48:26.334813 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:26.334312    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerDied","Data":"13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79"}
Jul 16 21:48:26.334813 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:26.334411    2582 scope.go:117] "RemoveContainer" containerID="3a107e9a858fa873350acea230a86f863dda95b6803aa25b38b5d70f84c34ae9"
Jul 16 21:48:26.334813 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:26.334659    2582 scope.go:117] "RemoveContainer" containerID="13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79"
Jul 16 21:48:26.334995 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:48:26.334880    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:48:27.341213 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:27.341187    2582 scope.go:117] "RemoveContainer" containerID="13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79"
Jul 16 21:48:27.341595 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:48:27.341382    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:48:29.198268 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:29.198217    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:48:29.198688 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:29.198631    2582 scope.go:117] "RemoveContainer" containerID="13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79"
Jul 16 21:48:29.198859 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:48:29.198838    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:48:33.254563 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254526    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"]
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254861    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerName="revoke-keys"
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254873    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerName="revoke-keys"
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254886    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerName="revoke-keys"
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254892    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerName="revoke-keys"
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254901    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerName="revoke-keys"
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254908    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerName="revoke-keys"
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254919    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerName="revoke-keys"
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254925    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerName="revoke-keys"
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254935    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerName="revoke-keys"
Jul 16 21:48:33.254947 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254941    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerName="revoke-keys"
Jul 16 21:48:33.255285 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.254998    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerName="revoke-keys"
Jul 16 21:48:33.255285 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.255007    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerName="revoke-keys"
Jul 16 21:48:33.255285 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.255013    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerName="revoke-keys"
Jul 16 21:48:33.255285 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.255019    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerName="revoke-keys"
Jul 16 21:48:33.255285 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.255023    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerName="revoke-keys"
Jul 16 21:48:33.259301 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.259284    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"
Jul 16 21:48:33.262462 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.262443    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:48:33.268269 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.268248    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"]
Jul 16 21:48:33.287897 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.287866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a106046-b68b-4fca-81a2-64a826b32a1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-46p72\" (UID: \"3a106046-b68b-4fca-81a2-64a826b32a1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"
Jul 16 21:48:33.388367 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.388333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a106046-b68b-4fca-81a2-64a826b32a1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-46p72\" (UID: \"3a106046-b68b-4fca-81a2-64a826b32a1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"
Jul 16 21:48:33.388990 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.388968    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a106046-b68b-4fca-81a2-64a826b32a1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-46p72\" (UID: \"3a106046-b68b-4fca-81a2-64a826b32a1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"
Jul 16 21:48:33.571904 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.571869    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"
Jul 16 21:48:33.716394 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:33.716370    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"]
Jul 16 21:48:33.718142 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:48:33.718114    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a106046_b68b_4fca_81a2_64a826b32a1b.slice/crio-9ddf29cfaf3945abc693a53503dd1cafe480bc68b80d515fc526cd97171ae6d1 WatchSource:0}: Error finding container 9ddf29cfaf3945abc693a53503dd1cafe480bc68b80d515fc526cd97171ae6d1: Status 404 returned error can't find the container with id 9ddf29cfaf3945abc693a53503dd1cafe480bc68b80d515fc526cd97171ae6d1
Jul 16 21:48:34.376629 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:34.376557    2582 generic.go:356] "Generic (PLEG): container finished" podID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerID="5dc11c54560cde9c886971cc2e8d23d73d9112493755668f1d0817e26ceab216" exitCode=7
Jul 16 21:48:34.377008 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:34.376703    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72" event={"ID":"3a106046-b68b-4fca-81a2-64a826b32a1b","Type":"ContainerDied","Data":"5dc11c54560cde9c886971cc2e8d23d73d9112493755668f1d0817e26ceab216"}
Jul 16 21:48:34.377008 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:34.376767    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72" event={"ID":"3a106046-b68b-4fca-81a2-64a826b32a1b","Type":"ContainerStarted","Data":"9ddf29cfaf3945abc693a53503dd1cafe480bc68b80d515fc526cd97171ae6d1"}
Jul 16 21:48:34.377008 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:34.376865    2582 scope.go:117] "RemoveContainer" containerID="5dc11c54560cde9c886971cc2e8d23d73d9112493755668f1d0817e26ceab216"
Jul 16 21:48:35.383828 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:35.383784    2582 generic.go:356] "Generic (PLEG): container finished" podID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerID="70ed8a1c8a68e006587b0b2489001d3282e19d1787e30991e1ffab64c05e3fe7" exitCode=7
Jul 16 21:48:35.384309 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:35.383870    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72" event={"ID":"3a106046-b68b-4fca-81a2-64a826b32a1b","Type":"ContainerDied","Data":"70ed8a1c8a68e006587b0b2489001d3282e19d1787e30991e1ffab64c05e3fe7"}
Jul 16 21:48:35.384309 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:35.383949    2582 scope.go:117] "RemoveContainer" containerID="5dc11c54560cde9c886971cc2e8d23d73d9112493755668f1d0817e26ceab216"
Jul 16 21:48:35.384309 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:35.384182    2582 scope.go:117] "RemoveContainer" containerID="70ed8a1c8a68e006587b0b2489001d3282e19d1787e30991e1ffab64c05e3fe7"
Jul 16 21:48:35.384474 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:48:35.384452    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-46p72_odh-ai-gateway-infra(3a106046-b68b-4fca-81a2-64a826b32a1b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b"
Jul 16 21:48:36.395749 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:36.395718    2582 scope.go:117] "RemoveContainer" containerID="70ed8a1c8a68e006587b0b2489001d3282e19d1787e30991e1ffab64c05e3fe7"
Jul 16 21:48:36.396255 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:48:36.395973    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-46p72_odh-ai-gateway-infra(3a106046-b68b-4fca-81a2-64a826b32a1b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b"
Jul 16 21:48:43.501562 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:43.501526    2582 scope.go:117] "RemoveContainer" containerID="13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79"
Jul 16 21:48:43.501941 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:48:43.501704    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:48:51.501350 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:51.501310    2582 scope.go:117] "RemoveContainer" containerID="70ed8a1c8a68e006587b0b2489001d3282e19d1787e30991e1ffab64c05e3fe7"
Jul 16 21:48:52.472308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:52.472221    2582 generic.go:356] "Generic (PLEG): container finished" podID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerID="70094c2cf4a1f53a9f6cb0f0d50806a375e0b871d608b74e89f8c007b6c58ab6" exitCode=7
Jul 16 21:48:52.472308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:52.472296    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72" event={"ID":"3a106046-b68b-4fca-81a2-64a826b32a1b","Type":"ContainerDied","Data":"70094c2cf4a1f53a9f6cb0f0d50806a375e0b871d608b74e89f8c007b6c58ab6"}
Jul 16 21:48:52.472510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:52.472359    2582 scope.go:117] "RemoveContainer" containerID="70ed8a1c8a68e006587b0b2489001d3282e19d1787e30991e1ffab64c05e3fe7"
Jul 16 21:48:52.472660 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:52.472638    2582 scope.go:117] "RemoveContainer" containerID="70094c2cf4a1f53a9f6cb0f0d50806a375e0b871d608b74e89f8c007b6c58ab6"
Jul 16 21:48:52.472873 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:48:52.472851    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-46p72_odh-ai-gateway-infra(3a106046-b68b-4fca-81a2-64a826b32a1b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b"
Jul 16 21:48:53.499448 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:53.499416    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"]
Jul 16 21:48:53.636695 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:53.636663    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"
Jul 16 21:48:53.763374 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:53.763282    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a106046-b68b-4fca-81a2-64a826b32a1b-maas-api-service-ca\") pod \"3a106046-b68b-4fca-81a2-64a826b32a1b\" (UID: \"3a106046-b68b-4fca-81a2-64a826b32a1b\") "
Jul 16 21:48:53.763675 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:53.763652    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a106046-b68b-4fca-81a2-64a826b32a1b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a106046-b68b-4fca-81a2-64a826b32a1b" (UID: "3a106046-b68b-4fca-81a2-64a826b32a1b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:48:53.864896 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:53.864858    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a106046-b68b-4fca-81a2-64a826b32a1b-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:48:54.484929 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:54.484901    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"
Jul 16 21:48:54.485164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:54.484896    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72" event={"ID":"3a106046-b68b-4fca-81a2-64a826b32a1b","Type":"ContainerDied","Data":"9ddf29cfaf3945abc693a53503dd1cafe480bc68b80d515fc526cd97171ae6d1"}
Jul 16 21:48:54.485164 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:54.485042    2582 scope.go:117] "RemoveContainer" containerID="70094c2cf4a1f53a9f6cb0f0d50806a375e0b871d608b74e89f8c007b6c58ab6"
Jul 16 21:48:54.507243 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:54.507218    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"]
Jul 16 21:48:54.513460 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:54.513433    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-46p72"]
Jul 16 21:48:55.501461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:55.501429    2582 scope.go:117] "RemoveContainer" containerID="13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79"
Jul 16 21:48:55.501746 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:48:55.501612    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:48:56.506138 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:48:56.506099    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" path="/var/lib/kubelet/pods/3a106046-b68b-4fca-81a2-64a826b32a1b/volumes"
Jul 16 21:49:04.404567 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.404532    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"]
Jul 16 21:49:04.404953 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.404877    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerName="revoke-keys"
Jul 16 21:49:04.404953 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.404887    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerName="revoke-keys"
Jul 16 21:49:04.404953 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.404907    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerName="revoke-keys"
Jul 16 21:49:04.404953 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.404912    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerName="revoke-keys"
Jul 16 21:49:04.404953 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.404921    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerName="revoke-keys"
Jul 16 21:49:04.404953 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.404928    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerName="revoke-keys"
Jul 16 21:49:04.404953 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.404940    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerName="revoke-keys"
Jul 16 21:49:04.404953 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.404945    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1497c096-c7b6-453d-bba0-d3eeebfd4d5f" containerName="revoke-keys"
Jul 16 21:49:04.405236 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.405006    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerName="revoke-keys"
Jul 16 21:49:04.405236 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.405014    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerName="revoke-keys"
Jul 16 21:49:04.405236 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.405022    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cd57344-76ca-44b7-8037-76e72bf8afb6" containerName="revoke-keys"
Jul 16 21:49:04.405236 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.405028    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a106046-b68b-4fca-81a2-64a826b32a1b" containerName="revoke-keys"
Jul 16 21:49:04.409412 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.409394    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"
Jul 16 21:49:04.411811 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.411788    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:49:04.416262 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.416235    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"]
Jul 16 21:49:04.560345 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.560298    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b51a67d-e338-4cfe-b254-0940a94c5a03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt\" (UID: \"6b51a67d-e338-4cfe-b254-0940a94c5a03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"
Jul 16 21:49:04.661903 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.661807    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b51a67d-e338-4cfe-b254-0940a94c5a03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt\" (UID: \"6b51a67d-e338-4cfe-b254-0940a94c5a03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"
Jul 16 21:49:04.662492 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.662470    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b51a67d-e338-4cfe-b254-0940a94c5a03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt\" (UID: \"6b51a67d-e338-4cfe-b254-0940a94c5a03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"
Jul 16 21:49:04.720786 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.720755    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"
Jul 16 21:49:04.856103 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:04.856057    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"]
Jul 16 21:49:04.859024 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:49:04.858987    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b51a67d_e338_4cfe_b254_0940a94c5a03.slice/crio-cda712b2f3b4934fa628908d233bcdaa162371b671fb7f43a7b3d05f3ec8bb36 WatchSource:0}: Error finding container cda712b2f3b4934fa628908d233bcdaa162371b671fb7f43a7b3d05f3ec8bb36: Status 404 returned error can't find the container with id cda712b2f3b4934fa628908d233bcdaa162371b671fb7f43a7b3d05f3ec8bb36
Jul 16 21:49:05.543133 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:05.543097    2582 generic.go:356] "Generic (PLEG): container finished" podID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerID="6865314e4a0a471e9a7b1845eb4ded03fcba79527c6bca168b1ce4ea5de82c03" exitCode=6
Jul 16 21:49:05.543612 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:05.543185    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt" event={"ID":"6b51a67d-e338-4cfe-b254-0940a94c5a03","Type":"ContainerDied","Data":"6865314e4a0a471e9a7b1845eb4ded03fcba79527c6bca168b1ce4ea5de82c03"}
Jul 16 21:49:05.543612 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:05.543241    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt" event={"ID":"6b51a67d-e338-4cfe-b254-0940a94c5a03","Type":"ContainerStarted","Data":"cda712b2f3b4934fa628908d233bcdaa162371b671fb7f43a7b3d05f3ec8bb36"}
Jul 16 21:49:05.543612 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:05.543390    2582 scope.go:117] "RemoveContainer" containerID="6865314e4a0a471e9a7b1845eb4ded03fcba79527c6bca168b1ce4ea5de82c03"
Jul 16 21:49:06.503144 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:06.503062    2582 scope.go:117] "RemoveContainer" containerID="13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79"
Jul 16 21:49:06.549510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:06.549476    2582 generic.go:356] "Generic (PLEG): container finished" podID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerID="8ef313aa0b04a8dda4134a9181a97b37274c5a0917a710d98dba6d29b1644ec8" exitCode=6
Jul 16 21:49:06.549973 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:06.549570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt" event={"ID":"6b51a67d-e338-4cfe-b254-0940a94c5a03","Type":"ContainerDied","Data":"8ef313aa0b04a8dda4134a9181a97b37274c5a0917a710d98dba6d29b1644ec8"}
Jul 16 21:49:06.549973 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:06.549643    2582 scope.go:117] "RemoveContainer" containerID="6865314e4a0a471e9a7b1845eb4ded03fcba79527c6bca168b1ce4ea5de82c03"
Jul 16 21:49:06.549973 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:06.549832    2582 scope.go:117] "RemoveContainer" containerID="8ef313aa0b04a8dda4134a9181a97b37274c5a0917a710d98dba6d29b1644ec8"
Jul 16 21:49:06.550185 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:06.550069    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt_odh-ai-gateway-infra(6b51a67d-e338-4cfe-b254-0940a94c5a03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03"
Jul 16 21:49:07.556380 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:07.556345    2582 generic.go:356] "Generic (PLEG): container finished" podID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241" exitCode=1
Jul 16 21:49:07.556839 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:07.556416    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerDied","Data":"cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"}
Jul 16 21:49:07.556839 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:07.556482    2582 scope.go:117] "RemoveContainer" containerID="13394b83d283932286e32d004173d852a9af891b5d1651baaa99112aaf61af79"
Jul 16 21:49:07.556839 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:07.556805    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:49:07.557059 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:07.557037    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:49:07.558236 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:07.558222    2582 scope.go:117] "RemoveContainer" containerID="8ef313aa0b04a8dda4134a9181a97b37274c5a0917a710d98dba6d29b1644ec8"
Jul 16 21:49:07.558444 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:07.558429    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt_odh-ai-gateway-infra(6b51a67d-e338-4cfe-b254-0940a94c5a03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03"
Jul 16 21:49:09.198838 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:09.198800    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:49:09.199265 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:09.199206    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:49:09.199406 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:09.199387    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:49:10.319807 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:10.319769    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:49:10.320340 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:10.320287    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:49:10.320519 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:10.320495    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:49:18.507239 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:18.507208    2582 scope.go:117] "RemoveContainer" containerID="8ef313aa0b04a8dda4134a9181a97b37274c5a0917a710d98dba6d29b1644ec8"
Jul 16 21:49:19.618197 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:19.618159    2582 generic.go:356] "Generic (PLEG): container finished" podID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerID="a22d565aba0452ff5c30f8ff904d2002b2db4c095ba3d5d07ff533c24bb70e53" exitCode=6
Jul 16 21:49:19.618632 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:19.618239    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt" event={"ID":"6b51a67d-e338-4cfe-b254-0940a94c5a03","Type":"ContainerDied","Data":"a22d565aba0452ff5c30f8ff904d2002b2db4c095ba3d5d07ff533c24bb70e53"}
Jul 16 21:49:19.618632 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:19.618305    2582 scope.go:117] "RemoveContainer" containerID="8ef313aa0b04a8dda4134a9181a97b37274c5a0917a710d98dba6d29b1644ec8"
Jul 16 21:49:19.618632 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:19.618575    2582 scope.go:117] "RemoveContainer" containerID="a22d565aba0452ff5c30f8ff904d2002b2db4c095ba3d5d07ff533c24bb70e53"
Jul 16 21:49:19.618806 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:19.618783    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt_odh-ai-gateway-infra(6b51a67d-e338-4cfe-b254-0940a94c5a03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03"
Jul 16 21:49:20.645045 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:20.645011    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"]
Jul 16 21:49:20.780426 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:20.780392    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"
Jul 16 21:49:20.818827 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:20.818798    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b51a67d-e338-4cfe-b254-0940a94c5a03-maas-api-service-ca\") pod \"6b51a67d-e338-4cfe-b254-0940a94c5a03\" (UID: \"6b51a67d-e338-4cfe-b254-0940a94c5a03\") "
Jul 16 21:49:20.819195 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:20.819172    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b51a67d-e338-4cfe-b254-0940a94c5a03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6b51a67d-e338-4cfe-b254-0940a94c5a03" (UID: "6b51a67d-e338-4cfe-b254-0940a94c5a03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:49:20.919590 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:20.919496    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b51a67d-e338-4cfe-b254-0940a94c5a03-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:49:21.630598 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:21.630560    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt" event={"ID":"6b51a67d-e338-4cfe-b254-0940a94c5a03","Type":"ContainerDied","Data":"cda712b2f3b4934fa628908d233bcdaa162371b671fb7f43a7b3d05f3ec8bb36"}
Jul 16 21:49:21.630598 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:21.630585    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"
Jul 16 21:49:21.630862 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:21.630633    2582 scope.go:117] "RemoveContainer" containerID="a22d565aba0452ff5c30f8ff904d2002b2db4c095ba3d5d07ff533c24bb70e53"
Jul 16 21:49:21.654610 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:21.654578    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"]
Jul 16 21:49:21.657707 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:21.657685    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-pdbwt"]
Jul 16 21:49:22.506782 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:22.506745    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" path="/var/lib/kubelet/pods/6b51a67d-e338-4cfe-b254-0940a94c5a03/volumes"
Jul 16 21:49:23.500774 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:23.500743    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:49:23.501194 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:23.500936    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:49:24.852972 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.852931    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"]
Jul 16 21:49:24.853518 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.853482    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerName="revoke-keys"
Jul 16 21:49:24.853518 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.853502    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerName="revoke-keys"
Jul 16 21:49:24.853633 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.853529    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerName="revoke-keys"
Jul 16 21:49:24.853633 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.853538    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerName="revoke-keys"
Jul 16 21:49:24.853633 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.853549    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerName="revoke-keys"
Jul 16 21:49:24.853633 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.853558    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerName="revoke-keys"
Jul 16 21:49:24.853821 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.853639    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerName="revoke-keys"
Jul 16 21:49:24.853821 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.853655    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerName="revoke-keys"
Jul 16 21:49:24.858560 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.858537    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"
Jul 16 21:49:24.861016 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.860997    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:49:24.866253 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.865878    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"]
Jul 16 21:49:24.960659 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:24.960615    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-2blch\" (UID: \"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"
Jul 16 21:49:25.061734 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:25.061689    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-2blch\" (UID: \"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"
Jul 16 21:49:25.062368 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:25.062349    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-2blch\" (UID: \"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"
Jul 16 21:49:25.170606 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:25.170520    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"
Jul 16 21:49:25.303149 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:25.303126    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"]
Jul 16 21:49:25.305686 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:49:25.305649    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3fe4b1_46d8_4020_8aa5_2ebb9e3dac98.slice/crio-43fce378be7b7e999f1701bfc7e3e50980e267a88872c45ad1a2213545ac10ff WatchSource:0}: Error finding container 43fce378be7b7e999f1701bfc7e3e50980e267a88872c45ad1a2213545ac10ff: Status 404 returned error can't find the container with id 43fce378be7b7e999f1701bfc7e3e50980e267a88872c45ad1a2213545ac10ff
Jul 16 21:49:25.650893 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:25.650862    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" event={"ID":"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98","Type":"ContainerStarted","Data":"dc3fdbc62b430b551449d33e329cc69e8e7fa5acce14376177540b78c822fd50"}
Jul 16 21:49:25.651093 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:25.650912    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" event={"ID":"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98","Type":"ContainerStarted","Data":"43fce378be7b7e999f1701bfc7e3e50980e267a88872c45ad1a2213545ac10ff"}
Jul 16 21:49:25.668897 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:25.668848    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" podStartSLOduration=1.6688323870000001 podStartE2EDuration="1.668832387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:25.666776854 +0000 UTC m=+2029.723607756" watchObservedRunningTime="2026-07-16 21:49:25.668832387 +0000 UTC m=+2029.725663299"
Jul 16 21:49:26.656600 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:26.656559    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerID="dc3fdbc62b430b551449d33e329cc69e8e7fa5acce14376177540b78c822fd50" exitCode=6
Jul 16 21:49:26.657066 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:26.656629    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" event={"ID":"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98","Type":"ContainerDied","Data":"dc3fdbc62b430b551449d33e329cc69e8e7fa5acce14376177540b78c822fd50"}
Jul 16 21:49:26.657066 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:26.656806    2582 scope.go:117] "RemoveContainer" containerID="dc3fdbc62b430b551449d33e329cc69e8e7fa5acce14376177540b78c822fd50"
Jul 16 21:49:27.664063 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:27.664028    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerID="e91020f3fbab6176234176323e9d2a3f8624c391c0925107c0f1dfa52c3dfa77" exitCode=6
Jul 16 21:49:27.664556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:27.664116    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" event={"ID":"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98","Type":"ContainerDied","Data":"e91020f3fbab6176234176323e9d2a3f8624c391c0925107c0f1dfa52c3dfa77"}
Jul 16 21:49:27.664556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:27.664178    2582 scope.go:117] "RemoveContainer" containerID="dc3fdbc62b430b551449d33e329cc69e8e7fa5acce14376177540b78c822fd50"
Jul 16 21:49:27.664556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:27.664404    2582 scope.go:117] "RemoveContainer" containerID="e91020f3fbab6176234176323e9d2a3f8624c391c0925107c0f1dfa52c3dfa77"
Jul 16 21:49:27.664731 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:27.664707    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-2blch_odh-ai-gateway-infra(7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98"
Jul 16 21:49:28.670952 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:28.670921    2582 scope.go:117] "RemoveContainer" containerID="e91020f3fbab6176234176323e9d2a3f8624c391c0925107c0f1dfa52c3dfa77"
Jul 16 21:49:28.671467 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:28.671226    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-2blch_odh-ai-gateway-infra(7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98"
Jul 16 21:49:38.500961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:38.500922    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:49:38.501472 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:38.501172    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:49:41.501741 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:41.501707    2582 scope.go:117] "RemoveContainer" containerID="e91020f3fbab6176234176323e9d2a3f8624c391c0925107c0f1dfa52c3dfa77"
Jul 16 21:49:41.734380 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:41.734352    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" event={"ID":"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98","Type":"ContainerStarted","Data":"bd299dfb2f6385fee8792cf7e2056d6c0ca6f0e506594d1e7af6161c1eb994cb"}
Jul 16 21:49:42.740827 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:42.740797    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerID="bd299dfb2f6385fee8792cf7e2056d6c0ca6f0e506594d1e7af6161c1eb994cb" exitCode=6
Jul 16 21:49:42.741249 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:42.740876    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" event={"ID":"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98","Type":"ContainerDied","Data":"bd299dfb2f6385fee8792cf7e2056d6c0ca6f0e506594d1e7af6161c1eb994cb"}
Jul 16 21:49:42.741249 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:42.740947    2582 scope.go:117] "RemoveContainer" containerID="e91020f3fbab6176234176323e9d2a3f8624c391c0925107c0f1dfa52c3dfa77"
Jul 16 21:49:42.741342 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:42.741262    2582 scope.go:117] "RemoveContainer" containerID="bd299dfb2f6385fee8792cf7e2056d6c0ca6f0e506594d1e7af6161c1eb994cb"
Jul 16 21:49:42.741500 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:42.741480    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-2blch_odh-ai-gateway-infra(7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98"
Jul 16 21:49:42.763753 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:42.763721    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"]
Jul 16 21:49:43.879786 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:43.879758    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"
Jul 16 21:49:44.033659 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:44.033620    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98-maas-api-service-ca\") pod \"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98\" (UID: \"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98\") "
Jul 16 21:49:44.034015 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:44.033981    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" (UID: "7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:49:44.134887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:44.134850    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:49:44.752803 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:44.752770    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"
Jul 16 21:49:44.752958 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:44.752770    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch" event={"ID":"7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98","Type":"ContainerDied","Data":"43fce378be7b7e999f1701bfc7e3e50980e267a88872c45ad1a2213545ac10ff"}
Jul 16 21:49:44.752958 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:44.752948    2582 scope.go:117] "RemoveContainer" containerID="bd299dfb2f6385fee8792cf7e2056d6c0ca6f0e506594d1e7af6161c1eb994cb"
Jul 16 21:49:44.770026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:44.769956    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"]
Jul 16 21:49:44.775846 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:44.775823    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-2blch"]
Jul 16 21:49:46.507163 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:46.507127    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" path="/var/lib/kubelet/pods/7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98/volumes"
Jul 16 21:49:52.501175 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:49:52.501143    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:49:52.501638 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:49:52.501300    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:50:07.501640 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:07.501601    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:50:07.502142 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:50:07.501870    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:50:14.901747 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.901717    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"]
Jul 16 21:50:14.902163 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.902112    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerName="revoke-keys"
Jul 16 21:50:14.902163 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.902125    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerName="revoke-keys"
Jul 16 21:50:14.902163 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.902148    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerName="revoke-keys"
Jul 16 21:50:14.902163 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.902154    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerName="revoke-keys"
Jul 16 21:50:14.902308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.902216    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerName="revoke-keys"
Jul 16 21:50:14.902308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.902227    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerName="revoke-keys"
Jul 16 21:50:14.902308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.902235    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b51a67d-e338-4cfe-b254-0940a94c5a03" containerName="revoke-keys"
Jul 16 21:50:14.902308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.902241    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerName="revoke-keys"
Jul 16 21:50:14.905212 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.905194    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"
Jul 16 21:50:14.908774 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.908754    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:50:14.918520 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:14.918495    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"]
Jul 16 21:50:15.026091 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:15.026050    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30f1c6e-1980-4577-bafc-856c8fefdc20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk\" (UID: \"c30f1c6e-1980-4577-bafc-856c8fefdc20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"
Jul 16 21:50:15.126818 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:15.126775    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30f1c6e-1980-4577-bafc-856c8fefdc20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk\" (UID: \"c30f1c6e-1980-4577-bafc-856c8fefdc20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"
Jul 16 21:50:15.127450 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:15.127427    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30f1c6e-1980-4577-bafc-856c8fefdc20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk\" (UID: \"c30f1c6e-1980-4577-bafc-856c8fefdc20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"
Jul 16 21:50:15.214862 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:15.214774    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"
Jul 16 21:50:15.353891 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:15.353860    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"]
Jul 16 21:50:15.356346 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:50:15.356316    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30f1c6e_1980_4577_bafc_856c8fefdc20.slice/crio-1f6dd597b8369c153110037614125b8ea1cedbc2a41aa0f0b6ffa0288f4f8575 WatchSource:0}: Error finding container 1f6dd597b8369c153110037614125b8ea1cedbc2a41aa0f0b6ffa0288f4f8575: Status 404 returned error can't find the container with id 1f6dd597b8369c153110037614125b8ea1cedbc2a41aa0f0b6ffa0288f4f8575
Jul 16 21:50:15.898769 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:15.898668    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" event={"ID":"c30f1c6e-1980-4577-bafc-856c8fefdc20","Type":"ContainerStarted","Data":"936ffd59de3c9cd6d25aa9e3f9519a88364abc7a78d4425863b1fa94590164d3"}
Jul 16 21:50:15.898769 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:15.898723    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" event={"ID":"c30f1c6e-1980-4577-bafc-856c8fefdc20","Type":"ContainerStarted","Data":"1f6dd597b8369c153110037614125b8ea1cedbc2a41aa0f0b6ffa0288f4f8575"}
Jul 16 21:50:15.915837 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:15.915787    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" podStartSLOduration=1.9157714289999999 podStartE2EDuration="1.915771429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:15.914686948 +0000 UTC m=+2079.971517861" watchObservedRunningTime="2026-07-16 21:50:15.915771429 +0000 UTC m=+2079.972602371"
Jul 16 21:50:16.905145 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:16.905111    2582 generic.go:356] "Generic (PLEG): container finished" podID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerID="936ffd59de3c9cd6d25aa9e3f9519a88364abc7a78d4425863b1fa94590164d3" exitCode=6
Jul 16 21:50:16.905316 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:16.905216    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" event={"ID":"c30f1c6e-1980-4577-bafc-856c8fefdc20","Type":"ContainerDied","Data":"936ffd59de3c9cd6d25aa9e3f9519a88364abc7a78d4425863b1fa94590164d3"}
Jul 16 21:50:16.905538 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:16.905518    2582 scope.go:117] "RemoveContainer" containerID="936ffd59de3c9cd6d25aa9e3f9519a88364abc7a78d4425863b1fa94590164d3"
Jul 16 21:50:17.912340 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:17.912250    2582 generic.go:356] "Generic (PLEG): container finished" podID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerID="7ba2c9ebdb79534d2622f027df390c603315c17775ba7699dcab580c48476355" exitCode=6
Jul 16 21:50:17.912340 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:17.912330    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" event={"ID":"c30f1c6e-1980-4577-bafc-856c8fefdc20","Type":"ContainerDied","Data":"7ba2c9ebdb79534d2622f027df390c603315c17775ba7699dcab580c48476355"}
Jul 16 21:50:17.912808 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:17.912386    2582 scope.go:117] "RemoveContainer" containerID="936ffd59de3c9cd6d25aa9e3f9519a88364abc7a78d4425863b1fa94590164d3"
Jul 16 21:50:17.912808 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:17.912620    2582 scope.go:117] "RemoveContainer" containerID="7ba2c9ebdb79534d2622f027df390c603315c17775ba7699dcab580c48476355"
Jul 16 21:50:17.912894 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:50:17.912818    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk_odh-ai-gateway-infra(c30f1c6e-1980-4577-bafc-856c8fefdc20)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20"
Jul 16 21:50:18.919966 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:18.919943    2582 scope.go:117] "RemoveContainer" containerID="7ba2c9ebdb79534d2622f027df390c603315c17775ba7699dcab580c48476355"
Jul 16 21:50:18.920445 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:50:18.920164    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk_odh-ai-gateway-infra(c30f1c6e-1980-4577-bafc-856c8fefdc20)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20"
Jul 16 21:50:22.501871 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:22.501829    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:50:22.502430 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:50:22.502012    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:50:26.283894 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:26.283862    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:33.501524 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:33.501490    2582 scope.go:117] "RemoveContainer" containerID="7ba2c9ebdb79534d2622f027df390c603315c17775ba7699dcab580c48476355"
Jul 16 21:50:33.995373 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:33.995340    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" event={"ID":"c30f1c6e-1980-4577-bafc-856c8fefdc20","Type":"ContainerStarted","Data":"4201a7426d3f6aace5fdb23badfc5fd1f631a1164e8927e08731a65c09754838"}
Jul 16 21:50:35.001514 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:35.001479    2582 generic.go:356] "Generic (PLEG): container finished" podID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerID="4201a7426d3f6aace5fdb23badfc5fd1f631a1164e8927e08731a65c09754838" exitCode=6
Jul 16 21:50:35.001938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:35.001554    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" event={"ID":"c30f1c6e-1980-4577-bafc-856c8fefdc20","Type":"ContainerDied","Data":"4201a7426d3f6aace5fdb23badfc5fd1f631a1164e8927e08731a65c09754838"}
Jul 16 21:50:35.001938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:35.001614    2582 scope.go:117] "RemoveContainer" containerID="7ba2c9ebdb79534d2622f027df390c603315c17775ba7699dcab580c48476355"
Jul 16 21:50:35.001938 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:35.001935    2582 scope.go:117] "RemoveContainer" containerID="4201a7426d3f6aace5fdb23badfc5fd1f631a1164e8927e08731a65c09754838"
Jul 16 21:50:35.002230 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:50:35.002202    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk_odh-ai-gateway-infra(c30f1c6e-1980-4577-bafc-856c8fefdc20)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20"
Jul 16 21:50:35.028428 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:35.028404    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"]
Jul 16 21:50:36.141367 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:36.141341    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"
Jul 16 21:50:36.331133 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:36.331102    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30f1c6e-1980-4577-bafc-856c8fefdc20-maas-api-service-ca\") pod \"c30f1c6e-1980-4577-bafc-856c8fefdc20\" (UID: \"c30f1c6e-1980-4577-bafc-856c8fefdc20\") "
Jul 16 21:50:36.331509 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:36.331486    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c30f1c6e-1980-4577-bafc-856c8fefdc20-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c30f1c6e-1980-4577-bafc-856c8fefdc20" (UID: "c30f1c6e-1980-4577-bafc-856c8fefdc20"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:50:36.432263 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:36.432227    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30f1c6e-1980-4577-bafc-856c8fefdc20-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:50:37.013725 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:37.013638    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk" event={"ID":"c30f1c6e-1980-4577-bafc-856c8fefdc20","Type":"ContainerDied","Data":"1f6dd597b8369c153110037614125b8ea1cedbc2a41aa0f0b6ffa0288f4f8575"}
Jul 16 21:50:37.013725 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:37.013665    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"
Jul 16 21:50:37.013725 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:37.013700    2582 scope.go:117] "RemoveContainer" containerID="4201a7426d3f6aace5fdb23badfc5fd1f631a1164e8927e08731a65c09754838"
Jul 16 21:50:37.033940 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:37.033909    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"]
Jul 16 21:50:37.037810 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:37.037785    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-njzrk"]
Jul 16 21:50:37.501468 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:37.501434    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:50:38.506207 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:38.506170    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" path="/var/lib/kubelet/pods/c30f1c6e-1980-4577-bafc-856c8fefdc20/volumes"
Jul 16 21:50:39.026246 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:39.026213    2582 generic.go:356] "Generic (PLEG): container finished" podID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756" exitCode=1
Jul 16 21:50:39.026439 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:39.026281    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerDied","Data":"582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"}
Jul 16 21:50:39.026439 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:39.026341    2582 scope.go:117] "RemoveContainer" containerID="cb25c9debd73b80ab9f98e7e877cb02108d1d0f6418ce52ad30846e792041241"
Jul 16 21:50:39.026716 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:39.026701    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:50:39.026921 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:50:39.026904    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:50:39.198908 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:39.198870    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:50:40.033302 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:40.033275    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:50:40.033687 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:50:40.033476    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:50:40.319881 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:40.319800    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:50:41.037716 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:41.037685    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:50:41.038130 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:50:41.037908    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:50:55.501158 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:55.501125    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:50:55.501546 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:50:55.501316    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:50:59.460831 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:50:59.460795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:01.016273 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:01.016233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:04.294622 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:04.294590    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:04.401793 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:04.401759    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:07.500896 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:07.500867    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:51:07.501296 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:07.501039    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:51:14.963013 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.962969    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"]
Jul 16 21:51:14.963531 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.963486    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerName="revoke-keys"
Jul 16 21:51:14.963531 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.963505    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerName="revoke-keys"
Jul 16 21:51:14.963656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.963538    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerName="revoke-keys"
Jul 16 21:51:14.963656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.963547    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3fe4b1-46d8-4020-8aa5-2ebb9e3dac98" containerName="revoke-keys"
Jul 16 21:51:14.963656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.963567    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerName="revoke-keys"
Jul 16 21:51:14.963656 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.963577    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerName="revoke-keys"
Jul 16 21:51:14.963846 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.963677    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerName="revoke-keys"
Jul 16 21:51:14.963846 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.963689    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerName="revoke-keys"
Jul 16 21:51:14.963846 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.963700    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerName="revoke-keys"
Jul 16 21:51:14.966622 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.966601    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"
Jul 16 21:51:14.969345 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.969321    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:51:14.976356 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:14.976330    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"]
Jul 16 21:51:15.075207 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:15.075161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk\" (UID: \"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"
Jul 16 21:51:15.176541 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:15.176494    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk\" (UID: \"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"
Jul 16 21:51:15.177204 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:15.177179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk\" (UID: \"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"
Jul 16 21:51:15.277212 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:15.277185    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"
Jul 16 21:51:15.426067 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:15.426039    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"]
Jul 16 21:51:15.428678 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:51:15.428646    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f0c1e42_4cb2_4f9c_b7f1_0ed27c4bcd84.slice/crio-4c8415880d3ecfb5e6edcaa81d38677a55094c7a560952101161e63561cd4a14 WatchSource:0}: Error finding container 4c8415880d3ecfb5e6edcaa81d38677a55094c7a560952101161e63561cd4a14: Status 404 returned error can't find the container with id 4c8415880d3ecfb5e6edcaa81d38677a55094c7a560952101161e63561cd4a14
Jul 16 21:51:16.200647 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:16.200605    2582 generic.go:356] "Generic (PLEG): container finished" podID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerID="2a57d9a86dfa7d9bfc987e6a31fd07dbe88a2e46b25ddb4523fb065b0410d1a5" exitCode=6
Jul 16 21:51:16.201053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:16.200694    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk" event={"ID":"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84","Type":"ContainerDied","Data":"2a57d9a86dfa7d9bfc987e6a31fd07dbe88a2e46b25ddb4523fb065b0410d1a5"}
Jul 16 21:51:16.201053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:16.200745    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk" event={"ID":"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84","Type":"ContainerStarted","Data":"4c8415880d3ecfb5e6edcaa81d38677a55094c7a560952101161e63561cd4a14"}
Jul 16 21:51:16.201053 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:16.200840    2582 scope.go:117] "RemoveContainer" containerID="2a57d9a86dfa7d9bfc987e6a31fd07dbe88a2e46b25ddb4523fb065b0410d1a5"
Jul 16 21:51:17.206575 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:17.206482    2582 generic.go:356] "Generic (PLEG): container finished" podID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerID="2c2d27b54f548401c01ae67ef375fb4334d40004d70ddaed24f28dff5e9ce43e" exitCode=6
Jul 16 21:51:17.206985 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:17.206609    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk" event={"ID":"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84","Type":"ContainerDied","Data":"2c2d27b54f548401c01ae67ef375fb4334d40004d70ddaed24f28dff5e9ce43e"}
Jul 16 21:51:17.206985 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:17.206687    2582 scope.go:117] "RemoveContainer" containerID="2a57d9a86dfa7d9bfc987e6a31fd07dbe88a2e46b25ddb4523fb065b0410d1a5"
Jul 16 21:51:17.206985 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:17.206752    2582 scope.go:117] "RemoveContainer" containerID="2c2d27b54f548401c01ae67ef375fb4334d40004d70ddaed24f28dff5e9ce43e"
Jul 16 21:51:17.207184 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:17.206988    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk_odh-ai-gateway-infra(4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84"
Jul 16 21:51:18.212814 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:18.212781    2582 scope.go:117] "RemoveContainer" containerID="2c2d27b54f548401c01ae67ef375fb4334d40004d70ddaed24f28dff5e9ce43e"
Jul 16 21:51:18.213275 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:18.212985    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk_odh-ai-gateway-infra(4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84"
Jul 16 21:51:18.501333 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:18.501254    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:51:18.501489 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:18.501439    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:51:31.501149 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:31.501052    2582 scope.go:117] "RemoveContainer" containerID="2c2d27b54f548401c01ae67ef375fb4334d40004d70ddaed24f28dff5e9ce43e"
Jul 16 21:51:32.279167 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:32.279135    2582 generic.go:356] "Generic (PLEG): container finished" podID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerID="144da91588ad22e99cfab5d64c898c4504635f274a98405533a50f745d7b7e18" exitCode=6
Jul 16 21:51:32.279341 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:32.279209    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk" event={"ID":"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84","Type":"ContainerDied","Data":"144da91588ad22e99cfab5d64c898c4504635f274a98405533a50f745d7b7e18"}
Jul 16 21:51:32.279341 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:32.279272    2582 scope.go:117] "RemoveContainer" containerID="2c2d27b54f548401c01ae67ef375fb4334d40004d70ddaed24f28dff5e9ce43e"
Jul 16 21:51:32.279542 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:32.279525    2582 scope.go:117] "RemoveContainer" containerID="144da91588ad22e99cfab5d64c898c4504635f274a98405533a50f745d7b7e18"
Jul 16 21:51:32.279778 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:32.279757    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk_odh-ai-gateway-infra(4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84"
Jul 16 21:51:33.318887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:33.318846    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"]
Jul 16 21:51:33.461994 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:33.461961    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"
Jul 16 21:51:33.501227 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:33.501200    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:51:33.501410 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:33.501391    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:51:33.550465 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:33.550434    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84-maas-api-service-ca\") pod \"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84\" (UID: \"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84\") "
Jul 16 21:51:33.550875 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:33.550849    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" (UID: "4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:51:33.652123 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:33.652008    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:51:34.293212 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:34.293183    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"
Jul 16 21:51:34.293393 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:34.293229    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk" event={"ID":"4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84","Type":"ContainerDied","Data":"4c8415880d3ecfb5e6edcaa81d38677a55094c7a560952101161e63561cd4a14"}
Jul 16 21:51:34.293393 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:34.293288    2582 scope.go:117] "RemoveContainer" containerID="144da91588ad22e99cfab5d64c898c4504635f274a98405533a50f745d7b7e18"
Jul 16 21:51:34.330032 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:34.329994    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"]
Jul 16 21:51:34.337722 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:34.337695    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lmlrk"]
Jul 16 21:51:34.506881 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:34.506851    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" path="/var/lib/kubelet/pods/4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84/volumes"
Jul 16 21:51:45.501092 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:45.501051    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:51:45.501483 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:45.501244    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:51:54.579999 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.579966    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"]
Jul 16 21:51:54.580414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580319    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerName="revoke-keys"
Jul 16 21:51:54.580414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580331    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerName="revoke-keys"
Jul 16 21:51:54.580414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580339    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerName="revoke-keys"
Jul 16 21:51:54.580414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580345    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerName="revoke-keys"
Jul 16 21:51:54.580414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580351    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerName="revoke-keys"
Jul 16 21:51:54.580414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580356    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerName="revoke-keys"
Jul 16 21:51:54.580414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580365    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerName="revoke-keys"
Jul 16 21:51:54.580414 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580371    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30f1c6e-1980-4577-bafc-856c8fefdc20" containerName="revoke-keys"
Jul 16 21:51:54.580664 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580453    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerName="revoke-keys"
Jul 16 21:51:54.580664 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.580460    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerName="revoke-keys"
Jul 16 21:51:54.583654 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.583638    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"
Jul 16 21:51:54.587331 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.587313    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:51:54.599066 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.599039    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"]
Jul 16 21:51:54.651853 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.651814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f273e96-d2e9-40aa-9676-c6b3657bec43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg\" (UID: \"8f273e96-d2e9-40aa-9676-c6b3657bec43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"
Jul 16 21:51:54.752325 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.752288    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f273e96-d2e9-40aa-9676-c6b3657bec43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg\" (UID: \"8f273e96-d2e9-40aa-9676-c6b3657bec43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"
Jul 16 21:51:54.752909 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.752887    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f273e96-d2e9-40aa-9676-c6b3657bec43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg\" (UID: \"8f273e96-d2e9-40aa-9676-c6b3657bec43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"
Jul 16 21:51:54.893716 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:54.893604    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"
Jul 16 21:51:55.028510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:55.028483    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"]
Jul 16 21:51:55.031258 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:51:55.031220    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f273e96_d2e9_40aa_9676_c6b3657bec43.slice/crio-c645e5648907fb50f564dab5841c34cc198dd19cd05fc758bc30dce772adf043 WatchSource:0}: Error finding container c645e5648907fb50f564dab5841c34cc198dd19cd05fc758bc30dce772adf043: Status 404 returned error can't find the container with id c645e5648907fb50f564dab5841c34cc198dd19cd05fc758bc30dce772adf043
Jul 16 21:51:55.400807 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:55.400769    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" event={"ID":"8f273e96-d2e9-40aa-9676-c6b3657bec43","Type":"ContainerStarted","Data":"5e4da71deb51087acde95ee9cc4a428e122420eab2e581d9f781b1e64a921c98"}
Jul 16 21:51:55.401013 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:55.400832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" event={"ID":"8f273e96-d2e9-40aa-9676-c6b3657bec43","Type":"ContainerStarted","Data":"c645e5648907fb50f564dab5841c34cc198dd19cd05fc758bc30dce772adf043"}
Jul 16 21:51:55.421559 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:55.421497    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" podStartSLOduration=1.421480511 podStartE2EDuration="1.421480511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:55.419687411 +0000 UTC m=+2179.476518323" watchObservedRunningTime="2026-07-16 21:51:55.421480511 +0000 UTC m=+2179.478311422"
Jul 16 21:51:56.407399 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:56.407362    2582 generic.go:356] "Generic (PLEG): container finished" podID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerID="5e4da71deb51087acde95ee9cc4a428e122420eab2e581d9f781b1e64a921c98" exitCode=6
Jul 16 21:51:56.407804 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:56.407417    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" event={"ID":"8f273e96-d2e9-40aa-9676-c6b3657bec43","Type":"ContainerDied","Data":"5e4da71deb51087acde95ee9cc4a428e122420eab2e581d9f781b1e64a921c98"}
Jul 16 21:51:56.407804 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:56.407666    2582 scope.go:117] "RemoveContainer" containerID="5e4da71deb51087acde95ee9cc4a428e122420eab2e581d9f781b1e64a921c98"
Jul 16 21:51:57.413851 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:57.413759    2582 generic.go:356] "Generic (PLEG): container finished" podID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerID="1907f786e96e25f7ff6c88e2d8c7acfce33dcf2bb3d5843d58f671992f17ae26" exitCode=6
Jul 16 21:51:57.413851 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:57.413822    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" event={"ID":"8f273e96-d2e9-40aa-9676-c6b3657bec43","Type":"ContainerDied","Data":"1907f786e96e25f7ff6c88e2d8c7acfce33dcf2bb3d5843d58f671992f17ae26"}
Jul 16 21:51:57.414343 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:57.413909    2582 scope.go:117] "RemoveContainer" containerID="5e4da71deb51087acde95ee9cc4a428e122420eab2e581d9f781b1e64a921c98"
Jul 16 21:51:57.414343 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:57.414125    2582 scope.go:117] "RemoveContainer" containerID="1907f786e96e25f7ff6c88e2d8c7acfce33dcf2bb3d5843d58f671992f17ae26"
Jul 16 21:51:57.414422 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:57.414371    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg_odh-ai-gateway-infra(8f273e96-d2e9-40aa-9676-c6b3657bec43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43"
Jul 16 21:51:57.501400 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:57.501368    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:51:57.501573 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:57.501539    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:51:58.420046 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:51:58.420011    2582 scope.go:117] "RemoveContainer" containerID="1907f786e96e25f7ff6c88e2d8c7acfce33dcf2bb3d5843d58f671992f17ae26"
Jul 16 21:51:58.420435 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:51:58.420265    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg_odh-ai-gateway-infra(8f273e96-d2e9-40aa-9676-c6b3657bec43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43"
Jul 16 21:52:10.500936 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:10.500898    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:52:10.501583 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:52:10.501100    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:52:13.501718 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:13.501690    2582 scope.go:117] "RemoveContainer" containerID="1907f786e96e25f7ff6c88e2d8c7acfce33dcf2bb3d5843d58f671992f17ae26"
Jul 16 21:52:14.500872 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:14.500784    2582 generic.go:356] "Generic (PLEG): container finished" podID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerID="ed2772473e0e20a96b1a9eb15b5f3d75b4ce033b16280175d3788d9cd49fe3ec" exitCode=6
Jul 16 21:52:14.506136 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:14.506109    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" event={"ID":"8f273e96-d2e9-40aa-9676-c6b3657bec43","Type":"ContainerDied","Data":"ed2772473e0e20a96b1a9eb15b5f3d75b4ce033b16280175d3788d9cd49fe3ec"}
Jul 16 21:52:14.506551 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:14.506165    2582 scope.go:117] "RemoveContainer" containerID="1907f786e96e25f7ff6c88e2d8c7acfce33dcf2bb3d5843d58f671992f17ae26"
Jul 16 21:52:14.506551 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:14.506424    2582 scope.go:117] "RemoveContainer" containerID="ed2772473e0e20a96b1a9eb15b5f3d75b4ce033b16280175d3788d9cd49fe3ec"
Jul 16 21:52:14.506698 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:52:14.506678    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg_odh-ai-gateway-infra(8f273e96-d2e9-40aa-9676-c6b3657bec43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43"
Jul 16 21:52:15.534446 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:15.534412    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"]
Jul 16 21:52:15.675977 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:15.675949    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"
Jul 16 21:52:15.749419 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:15.749384    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f273e96-d2e9-40aa-9676-c6b3657bec43-maas-api-service-ca\") pod \"8f273e96-d2e9-40aa-9676-c6b3657bec43\" (UID: \"8f273e96-d2e9-40aa-9676-c6b3657bec43\") "
Jul 16 21:52:15.749763 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:15.749739    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f273e96-d2e9-40aa-9676-c6b3657bec43-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f273e96-d2e9-40aa-9676-c6b3657bec43" (UID: "8f273e96-d2e9-40aa-9676-c6b3657bec43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:52:15.851148 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:15.851023    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f273e96-d2e9-40aa-9676-c6b3657bec43-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:52:16.512090 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:16.512038    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg" event={"ID":"8f273e96-d2e9-40aa-9676-c6b3657bec43","Type":"ContainerDied","Data":"c645e5648907fb50f564dab5841c34cc198dd19cd05fc758bc30dce772adf043"}
Jul 16 21:52:16.512268 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:16.512122    2582 scope.go:117] "RemoveContainer" containerID="ed2772473e0e20a96b1a9eb15b5f3d75b4ce033b16280175d3788d9cd49fe3ec"
Jul 16 21:52:16.512268 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:16.512046    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"
Jul 16 21:52:16.535008 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:16.534982    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"]
Jul 16 21:52:16.541398 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:16.541371    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-fzlhg"]
Jul 16 21:52:18.506538 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:18.506500    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" path="/var/lib/kubelet/pods/8f273e96-d2e9-40aa-9676-c6b3657bec43/volumes"
Jul 16 21:52:25.501556 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:25.501522    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:52:25.503929 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:52:25.501718    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:52:40.501129 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:40.501068    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:52:40.501540 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:52:40.501341    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:52:54.635487 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635452    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"]
Jul 16 21:52:54.635961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635798    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerName="revoke-keys"
Jul 16 21:52:54.635961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635809    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerName="revoke-keys"
Jul 16 21:52:54.635961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635818    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerName="revoke-keys"
Jul 16 21:52:54.635961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635824    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerName="revoke-keys"
Jul 16 21:52:54.635961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635840    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerName="revoke-keys"
Jul 16 21:52:54.635961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635846    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerName="revoke-keys"
Jul 16 21:52:54.635961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635918    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f0c1e42-4cb2-4f9c-b7f1-0ed27c4bcd84" containerName="revoke-keys"
Jul 16 21:52:54.635961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635927    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerName="revoke-keys"
Jul 16 21:52:54.635961 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.635935    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerName="revoke-keys"
Jul 16 21:52:54.639031 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.639011    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"
Jul 16 21:52:54.643024 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.643006    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:52:54.649323 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.649297    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"]
Jul 16 21:52:54.709294 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.709258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b470be-e037-425a-98cc-df5796856850-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq\" (UID: \"d2b470be-e037-425a-98cc-df5796856850\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"
Jul 16 21:52:54.809752 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.809701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b470be-e037-425a-98cc-df5796856850-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq\" (UID: \"d2b470be-e037-425a-98cc-df5796856850\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"
Jul 16 21:52:54.810358 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.810337    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b470be-e037-425a-98cc-df5796856850-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq\" (UID: \"d2b470be-e037-425a-98cc-df5796856850\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"
Jul 16 21:52:54.949264 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:54.949150    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"
Jul 16 21:52:55.091609 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:55.091493    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"]
Jul 16 21:52:55.094352 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:52:55.094323    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b470be_e037_425a_98cc_df5796856850.slice/crio-ddfa60d849039db8d771f18d3347f964bd6aa2109b9afebfff66aebbd9693e18 WatchSource:0}: Error finding container ddfa60d849039db8d771f18d3347f964bd6aa2109b9afebfff66aebbd9693e18: Status 404 returned error can't find the container with id ddfa60d849039db8d771f18d3347f964bd6aa2109b9afebfff66aebbd9693e18
Jul 16 21:52:55.501557 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:55.501521    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:52:55.501755 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:52:55.501720    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:52:55.694330 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:55.694240    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" event={"ID":"d2b470be-e037-425a-98cc-df5796856850","Type":"ContainerStarted","Data":"b6fce475c50b86084786c5ba891fa5780ab9a08fe3c35c45891a7517b09dbed9"}
Jul 16 21:52:55.694330 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:55.694297    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" event={"ID":"d2b470be-e037-425a-98cc-df5796856850","Type":"ContainerStarted","Data":"ddfa60d849039db8d771f18d3347f964bd6aa2109b9afebfff66aebbd9693e18"}
Jul 16 21:52:55.714859 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:55.714799    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" podStartSLOduration=1.71477851 podStartE2EDuration="1.71477851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:55.713723005 +0000 UTC m=+2239.770553917" watchObservedRunningTime="2026-07-16 21:52:55.71477851 +0000 UTC m=+2239.771609423"
Jul 16 21:52:56.701475 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:56.701392    2582 generic.go:356] "Generic (PLEG): container finished" podID="d2b470be-e037-425a-98cc-df5796856850" containerID="b6fce475c50b86084786c5ba891fa5780ab9a08fe3c35c45891a7517b09dbed9" exitCode=6
Jul 16 21:52:56.701916 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:56.701487    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" event={"ID":"d2b470be-e037-425a-98cc-df5796856850","Type":"ContainerDied","Data":"b6fce475c50b86084786c5ba891fa5780ab9a08fe3c35c45891a7517b09dbed9"}
Jul 16 21:52:56.701916 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:56.701672    2582 scope.go:117] "RemoveContainer" containerID="b6fce475c50b86084786c5ba891fa5780ab9a08fe3c35c45891a7517b09dbed9"
Jul 16 21:52:57.707732 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:57.707640    2582 generic.go:356] "Generic (PLEG): container finished" podID="d2b470be-e037-425a-98cc-df5796856850" containerID="6d45559cb898c19b4080003f78aa58cd1f7fbd030c4460e107761ae8068abee1" exitCode=6
Jul 16 21:52:57.708136 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:57.707738    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" event={"ID":"d2b470be-e037-425a-98cc-df5796856850","Type":"ContainerDied","Data":"6d45559cb898c19b4080003f78aa58cd1f7fbd030c4460e107761ae8068abee1"}
Jul 16 21:52:57.708136 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:57.707784    2582 scope.go:117] "RemoveContainer" containerID="b6fce475c50b86084786c5ba891fa5780ab9a08fe3c35c45891a7517b09dbed9"
Jul 16 21:52:57.708136 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:57.707909    2582 scope.go:117] "RemoveContainer" containerID="6d45559cb898c19b4080003f78aa58cd1f7fbd030c4460e107761ae8068abee1"
Jul 16 21:52:57.708280 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:52:57.708136    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq_odh-ai-gateway-infra(d2b470be-e037-425a-98cc-df5796856850)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" podUID="d2b470be-e037-425a-98cc-df5796856850"
Jul 16 21:52:58.714030 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:52:58.713997    2582 scope.go:117] "RemoveContainer" containerID="6d45559cb898c19b4080003f78aa58cd1f7fbd030c4460e107761ae8068abee1"
Jul 16 21:52:58.714444 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:52:58.714236    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq_odh-ai-gateway-infra(d2b470be-e037-425a-98cc-df5796856850)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" podUID="d2b470be-e037-425a-98cc-df5796856850"
Jul 16 21:53:07.211966 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:07.211926    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:07.501534 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:07.501450    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:53:07.501688 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:53:07.501617    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:53:13.501457 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:13.501423    2582 scope.go:117] "RemoveContainer" containerID="6d45559cb898c19b4080003f78aa58cd1f7fbd030c4460e107761ae8068abee1"
Jul 16 21:53:14.789553 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:14.789515    2582 generic.go:356] "Generic (PLEG): container finished" podID="d2b470be-e037-425a-98cc-df5796856850" containerID="23289043f6fc26a956c5535f681f2c1f11880a5ed0f94e2f3f5fd41fa31c7d22" exitCode=6
Jul 16 21:53:14.790066 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:14.789588    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" event={"ID":"d2b470be-e037-425a-98cc-df5796856850","Type":"ContainerDied","Data":"23289043f6fc26a956c5535f681f2c1f11880a5ed0f94e2f3f5fd41fa31c7d22"}
Jul 16 21:53:14.790066 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:14.789650    2582 scope.go:117] "RemoveContainer" containerID="6d45559cb898c19b4080003f78aa58cd1f7fbd030c4460e107761ae8068abee1"
Jul 16 21:53:14.790066 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:14.789894    2582 scope.go:117] "RemoveContainer" containerID="23289043f6fc26a956c5535f681f2c1f11880a5ed0f94e2f3f5fd41fa31c7d22"
Jul 16 21:53:14.790268 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:53:14.790135    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq_odh-ai-gateway-infra(d2b470be-e037-425a-98cc-df5796856850)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" podUID="d2b470be-e037-425a-98cc-df5796856850"
Jul 16 21:53:15.817162 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:15.817128    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"]
Jul 16 21:53:15.957517 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:15.957491    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"
Jul 16 21:53:15.997040 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:15.996995    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b470be-e037-425a-98cc-df5796856850-maas-api-service-ca\") pod \"d2b470be-e037-425a-98cc-df5796856850\" (UID: \"d2b470be-e037-425a-98cc-df5796856850\") "
Jul 16 21:53:15.997387 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:15.997360    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b470be-e037-425a-98cc-df5796856850-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d2b470be-e037-425a-98cc-df5796856850" (UID: "d2b470be-e037-425a-98cc-df5796856850"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:16.098310 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:16.098216    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b470be-e037-425a-98cc-df5796856850-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:53:16.803349 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:16.803322    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"
Jul 16 21:53:16.803555 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:16.803363    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq" event={"ID":"d2b470be-e037-425a-98cc-df5796856850","Type":"ContainerDied","Data":"ddfa60d849039db8d771f18d3347f964bd6aa2109b9afebfff66aebbd9693e18"}
Jul 16 21:53:16.803555 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:16.803427    2582 scope.go:117] "RemoveContainer" containerID="23289043f6fc26a956c5535f681f2c1f11880a5ed0f94e2f3f5fd41fa31c7d22"
Jul 16 21:53:16.826447 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:16.824828    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"]
Jul 16 21:53:16.832566 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:16.832527    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-c2mcq"]
Jul 16 21:53:18.500936 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:18.500906    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:53:18.501347 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:18.501238    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:53:18.506652 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:18.506626    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2b470be-e037-425a-98cc-df5796856850" path="/var/lib/kubelet/pods/d2b470be-e037-425a-98cc-df5796856850/volumes"
Jul 16 21:53:19.823232 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:19.823200    2582 generic.go:356] "Generic (PLEG): container finished" podID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerID="cfaf773e6a450f0ad7a72355bf88eb4c3ad07ff3305abc75320f1960014a7974" exitCode=1
Jul 16 21:53:19.823694 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:19.823272    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerDied","Data":"cfaf773e6a450f0ad7a72355bf88eb4c3ad07ff3305abc75320f1960014a7974"}
Jul 16 21:53:19.823694 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:19.823348    2582 scope.go:117] "RemoveContainer" containerID="582f35d3b4ade680cf728cb060f374b0fbb49cc07e3774fb59dda203307a1756"
Jul 16 21:53:19.823782 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:19.823749    2582 scope.go:117] "RemoveContainer" containerID="cfaf773e6a450f0ad7a72355bf88eb4c3ad07ff3305abc75320f1960014a7974"
Jul 16 21:53:19.823983 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:53:19.823957    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:53:20.319824 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:20.319774    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:53:20.829758 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:20.829723    2582 scope.go:117] "RemoveContainer" containerID="cfaf773e6a450f0ad7a72355bf88eb4c3ad07ff3305abc75320f1960014a7974"
Jul 16 21:53:20.830259 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:53:20.829965    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:53:29.198820 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:29.198776    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:53:29.199361 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:29.199261    2582 scope.go:117] "RemoveContainer" containerID="cfaf773e6a450f0ad7a72355bf88eb4c3ad07ff3305abc75320f1960014a7974"
Jul 16 21:53:29.199492 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:53:29.199470    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd_odh-ai-gateway-infra(33fb0a43-a8ca-4838-931f-4e2919bab69e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e"
Jul 16 21:53:33.454507 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:33.454469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:33.561059 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:33.561024    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:39.494586 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.494549    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"]
Jul 16 21:53:39.521383 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.521348    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:39.632335 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.632311    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:53:39.717952 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.717918    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fb0a43-a8ca-4838-931f-4e2919bab69e-openshift-service-ca\") pod \"33fb0a43-a8ca-4838-931f-4e2919bab69e\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") "
Jul 16 21:53:39.718165 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.717972    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6sb8\" (UniqueName: \"kubernetes.io/projected/33fb0a43-a8ca-4838-931f-4e2919bab69e-kube-api-access-x6sb8\") pod \"33fb0a43-a8ca-4838-931f-4e2919bab69e\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") "
Jul 16 21:53:39.718372 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.718345    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fb0a43-a8ca-4838-931f-4e2919bab69e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33fb0a43-a8ca-4838-931f-4e2919bab69e" (UID: "33fb0a43-a8ca-4838-931f-4e2919bab69e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:39.720220 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.720185    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33fb0a43-a8ca-4838-931f-4e2919bab69e-kube-api-access-x6sb8" (OuterVolumeSpecName: "kube-api-access-x6sb8") pod "33fb0a43-a8ca-4838-931f-4e2919bab69e" (UID: "33fb0a43-a8ca-4838-931f-4e2919bab69e"). InnerVolumeSpecName "kube-api-access-x6sb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:53:39.819430 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.819398    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fb0a43-a8ca-4838-931f-4e2919bab69e-maas-api-tls\") pod \"33fb0a43-a8ca-4838-931f-4e2919bab69e\" (UID: \"33fb0a43-a8ca-4838-931f-4e2919bab69e\") "
Jul 16 21:53:39.819602 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.819584    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6sb8\" (UniqueName: \"kubernetes.io/projected/33fb0a43-a8ca-4838-931f-4e2919bab69e-kube-api-access-x6sb8\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:53:39.819602 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.819594    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fb0a43-a8ca-4838-931f-4e2919bab69e-openshift-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:53:39.821561 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.821526    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fb0a43-a8ca-4838-931f-4e2919bab69e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33fb0a43-a8ca-4838-931f-4e2919bab69e" (UID: "33fb0a43-a8ca-4838-931f-4e2919bab69e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:39.918924 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.918886    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd" event={"ID":"33fb0a43-a8ca-4838-931f-4e2919bab69e","Type":"ContainerDied","Data":"3e89f1e9f97f75c11d8d64f02a0b6d13ec3c78535fc96400132325d072d0c9aa"}
Jul 16 21:53:39.918924 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.918912    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"
Jul 16 21:53:39.919182 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.918965    2582 scope.go:117] "RemoveContainer" containerID="cfaf773e6a450f0ad7a72355bf88eb4c3ad07ff3305abc75320f1960014a7974"
Jul 16 21:53:39.920111 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.920038    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fb0a43-a8ca-4838-931f-4e2919bab69e-maas-api-tls\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:53:39.939532 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.939504    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"]
Jul 16 21:53:39.945419 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:39.945395    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6d0b514-57f88585bd-ftqcd"]
Jul 16 21:53:40.002539 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:40.002512    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:40.507570 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:40.507539    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" path="/var/lib/kubelet/pods/33fb0a43-a8ca-4838-931f-4e2919bab69e/volumes"
Jul 16 21:53:46.034734 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:46.034698    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:46.135297 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:46.135270    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:46.485826 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:46.485730    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76559f66cc-b9f6c_ce7121eb-d400-4f17-97d6-57bdcea6f7c9/authorino/0.log"
Jul 16 21:53:52.386801 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:52.386770    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2_255465d7-fbd4-4536-baf0-2a1b696f322f/copy-manifests/0.log"
Jul 16 21:53:52.394547 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:52.394517    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2_255465d7-fbd4-4536-baf0-2a1b696f322f/manager/0.log"
Jul 16 21:53:52.991952 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:52.991925    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7954f7d7ff-5mlzf_e400471d-2484-4358-8038-1ed16965929f/maas-api/0.log"
Jul 16 21:53:53.205345 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:53.205307    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737305-mqbgf_6c1701af-cd02-427d-9f9b-0ec0b6b8b0a1/cleanup/0.log"
Jul 16 21:53:54.669514 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:54.669480    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76559f66cc-b9f6c_ce7121eb-d400-4f17-97d6-57bdcea6f7c9/authorino/0.log"
Jul 16 21:53:54.888055 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:54.888022    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-b9fqb_0749f338-d140-47b4-b636-9df5ae09c1c5/manager/0.log"
Jul 16 21:53:54.993783 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:54.993705    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vk7hd_827eb8d6-61d7-4815-8054-f8e552de1eae/kuadrant-console-plugin/0.log"
Jul 16 21:53:55.255040 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:55.254960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:56.150631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:56.150601    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-796f7d4c6-wjjfl_d2e98803-3f5b-4e4b-b04f-9adfd5045a74/kube-auth-proxy/0.log"
Jul 16 21:53:56.399915 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:56.399879    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gdmsm_596dec7c-a5fb-4aae-8f2d-5903673b4eb4/payload-pre-processing/0.log"
Jul 16 21:53:56.515318 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:56.515235    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-7zlxf_42863b88-d149-4131-8598-6921b305389b/payload-processing/0.log"
Jul 16 21:53:57.205838 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:57.205805    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-kdqqv_78197794-2038-40cb-b27e-d68d794efd38/main/0.log"
Jul 16 21:53:57.323777 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:57.323742    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4vccn_4add1873-0d5a-4f52-b7ef-6659d151f4eb/main/0.log"
Jul 16 21:53:57.555641 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:53:57.555609    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-fglmp_c56257c8-22a7-4c81-9a13-336e256c6fec/main/0.log"
Jul 16 21:54:03.330608 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:03.330572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:03.400510 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:03.400477    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:05.120714 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.120676    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"]
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121050    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121062    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121071    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b470be-e037-425a-98cc-df5796856850" containerName="revoke-keys"
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121095    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b470be-e037-425a-98cc-df5796856850" containerName="revoke-keys"
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121113    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121122    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121134    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b470be-e037-425a-98cc-df5796856850" containerName="revoke-keys"
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121139    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b470be-e037-425a-98cc-df5796856850" containerName="revoke-keys"
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121145    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121146 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121150    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121160    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121167    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121172    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121177    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121234    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121240    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f273e96-d2e9-40aa-9676-c6b3657bec43" containerName="revoke-keys"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121249    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121254    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121258    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b470be-e037-425a-98cc-df5796856850" containerName="revoke-keys"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121265    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b470be-e037-425a-98cc-df5796856850" containerName="revoke-keys"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121270    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b470be-e037-425a-98cc-df5796856850" containerName="revoke-keys"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121277    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.121461 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.121282    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:05.125715 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.125695    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"
Jul 16 21:54:05.128051 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.128020    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:54:05.131646 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.131615    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"]
Jul 16 21:54:05.247931 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.247885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0191b7-4baf-4d7b-9c0c-e54a286b927e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c\" (UID: \"2f0191b7-4baf-4d7b-9c0c-e54a286b927e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"
Jul 16 21:54:05.349024 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.348987    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0191b7-4baf-4d7b-9c0c-e54a286b927e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c\" (UID: \"2f0191b7-4baf-4d7b-9c0c-e54a286b927e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"
Jul 16 21:54:05.349633 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.349613    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0191b7-4baf-4d7b-9c0c-e54a286b927e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c\" (UID: \"2f0191b7-4baf-4d7b-9c0c-e54a286b927e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"
Jul 16 21:54:05.438923 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.438828    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"
Jul 16 21:54:05.582288 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:05.582260    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"]
Jul 16 21:54:05.583984 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:54:05.583954    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f0191b7_4baf_4d7b_9c0c_e54a286b927e.slice/crio-340d86fc313e69a451551ccdb081963a1cc3ed5b84f36ce1abdd23ef5e0b98fe WatchSource:0}: Error finding container 340d86fc313e69a451551ccdb081963a1cc3ed5b84f36ce1abdd23ef5e0b98fe: Status 404 returned error can't find the container with id 340d86fc313e69a451551ccdb081963a1cc3ed5b84f36ce1abdd23ef5e0b98fe
Jul 16 21:54:06.039425 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:06.039381    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" event={"ID":"2f0191b7-4baf-4d7b-9c0c-e54a286b927e","Type":"ContainerStarted","Data":"9f92550f7df68731c25b4c19d5fbde5c0ab8ba2a5dc0e57d9129f645289eae85"}
Jul 16 21:54:06.039619 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:06.039443    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" event={"ID":"2f0191b7-4baf-4d7b-9c0c-e54a286b927e","Type":"ContainerStarted","Data":"340d86fc313e69a451551ccdb081963a1cc3ed5b84f36ce1abdd23ef5e0b98fe"}
Jul 16 21:54:06.059710 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:06.059655    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" podStartSLOduration=1.059639194 podStartE2EDuration="1.059639194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:06.057962748 +0000 UTC m=+2310.114793657" watchObservedRunningTime="2026-07-16 21:54:06.059639194 +0000 UTC m=+2310.116470105"
Jul 16 21:54:06.366995 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:06.366912    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h98vh_def9cc97-3b3b-455a-a095-21a14521bb12/global-pull-secret-syncer/0.log"
Jul 16 21:54:06.540720 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:06.540686    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-657qw_2781ca06-e247-4b8b-879e-72a32dbdc0bf/konnectivity-agent/0.log"
Jul 16 21:54:06.729834 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:06.729749    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-201.ec2.internal_33cd532730a76493a09bd77465fc98c4/haproxy/0.log"
Jul 16 21:54:07.046094 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:07.046044    2582 generic.go:356] "Generic (PLEG): container finished" podID="2f0191b7-4baf-4d7b-9c0c-e54a286b927e" containerID="9f92550f7df68731c25b4c19d5fbde5c0ab8ba2a5dc0e57d9129f645289eae85" exitCode=6
Jul 16 21:54:07.046312 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:07.046115    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" event={"ID":"2f0191b7-4baf-4d7b-9c0c-e54a286b927e","Type":"ContainerDied","Data":"9f92550f7df68731c25b4c19d5fbde5c0ab8ba2a5dc0e57d9129f645289eae85"}
Jul 16 21:54:07.046409 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:07.046393    2582 scope.go:117] "RemoveContainer" containerID="9f92550f7df68731c25b4c19d5fbde5c0ab8ba2a5dc0e57d9129f645289eae85"
Jul 16 21:54:08.052268 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:08.052232    2582 generic.go:356] "Generic (PLEG): container finished" podID="2f0191b7-4baf-4d7b-9c0c-e54a286b927e" containerID="25d83edc1456e11b0a951fed82c1f07365904928db597fdde4935641ec1ea6ef" exitCode=6
Jul 16 21:54:08.052762 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:08.052312    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" event={"ID":"2f0191b7-4baf-4d7b-9c0c-e54a286b927e","Type":"ContainerDied","Data":"25d83edc1456e11b0a951fed82c1f07365904928db597fdde4935641ec1ea6ef"}
Jul 16 21:54:08.052762 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:08.052366    2582 scope.go:117] "RemoveContainer" containerID="9f92550f7df68731c25b4c19d5fbde5c0ab8ba2a5dc0e57d9129f645289eae85"
Jul 16 21:54:08.052762 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:08.052590    2582 scope.go:117] "RemoveContainer" containerID="25d83edc1456e11b0a951fed82c1f07365904928db597fdde4935641ec1ea6ef"
Jul 16 21:54:08.052926 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:54:08.052846    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c_odh-ai-gateway-infra(2f0191b7-4baf-4d7b-9c0c-e54a286b927e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" podUID="2f0191b7-4baf-4d7b-9c0c-e54a286b927e"
Jul 16 21:54:09.058752 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:09.058720    2582 scope.go:117] "RemoveContainer" containerID="25d83edc1456e11b0a951fed82c1f07365904928db597fdde4935641ec1ea6ef"
Jul 16 21:54:09.059254 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:54:09.058981    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c_odh-ai-gateway-infra(2f0191b7-4baf-4d7b-9c0c-e54a286b927e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" podUID="2f0191b7-4baf-4d7b-9c0c-e54a286b927e"
Jul 16 21:54:12.319422 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:12.319382    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76559f66cc-b9f6c_ce7121eb-d400-4f17-97d6-57bdcea6f7c9/authorino/0.log"
Jul 16 21:54:12.419635 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:12.419587    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-b9fqb_0749f338-d140-47b4-b636-9df5ae09c1c5/manager/0.log"
Jul 16 21:54:12.451599 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:12.451572    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vk7hd_827eb8d6-61d7-4815-8054-f8e552de1eae/kuadrant-console-plugin/0.log"
Jul 16 21:54:14.491607 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:14.491576    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ec5e9f1-0527-4d99-bcdf-50b19b71f633/alertmanager/0.log"
Jul 16 21:54:14.520425 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:14.520391    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ec5e9f1-0527-4d99-bcdf-50b19b71f633/config-reloader/0.log"
Jul 16 21:54:14.569885 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:14.569857    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ec5e9f1-0527-4d99-bcdf-50b19b71f633/kube-rbac-proxy-web/0.log"
Jul 16 21:54:14.641040 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:14.640964    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ec5e9f1-0527-4d99-bcdf-50b19b71f633/kube-rbac-proxy/0.log"
Jul 16 21:54:14.682773 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:14.682748    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ec5e9f1-0527-4d99-bcdf-50b19b71f633/kube-rbac-proxy-metric/0.log"
Jul 16 21:54:14.724998 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:14.724965    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ec5e9f1-0527-4d99-bcdf-50b19b71f633/prom-label-proxy/0.log"
Jul 16 21:54:14.758508 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:14.758479    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ec5e9f1-0527-4d99-bcdf-50b19b71f633/init-config-reloader/0.log"
Jul 16 21:54:14.978887 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:14.978814    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-627gf_e3021cbf-7924-46fb-b841-196b3e61b20d/monitoring-plugin/0.log"
Jul 16 21:54:15.019793 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.019768    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d8f74_5a1ce953-a8b2-476c-863c-049f522071bf/node-exporter/0.log"
Jul 16 21:54:15.046033 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.046008    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d8f74_5a1ce953-a8b2-476c-863c-049f522071bf/kube-rbac-proxy/0.log"
Jul 16 21:54:15.085694 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.085670    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d8f74_5a1ce953-a8b2-476c-863c-049f522071bf/init-textfile/0.log"
Jul 16 21:54:15.538192 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.538164    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51fea4e7-4c47-40c7-9299-8b4351c1bf34/prometheus/0.log"
Jul 16 21:54:15.564926 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.564899    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51fea4e7-4c47-40c7-9299-8b4351c1bf34/config-reloader/0.log"
Jul 16 21:54:15.600903 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.600875    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51fea4e7-4c47-40c7-9299-8b4351c1bf34/thanos-sidecar/0.log"
Jul 16 21:54:15.626732 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.626703    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51fea4e7-4c47-40c7-9299-8b4351c1bf34/kube-rbac-proxy-web/0.log"
Jul 16 21:54:15.655898 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.655875    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51fea4e7-4c47-40c7-9299-8b4351c1bf34/kube-rbac-proxy/0.log"
Jul 16 21:54:15.681063 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.681030    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51fea4e7-4c47-40c7-9299-8b4351c1bf34/kube-rbac-proxy-thanos/0.log"
Jul 16 21:54:15.710816 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:15.710782    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51fea4e7-4c47-40c7-9299-8b4351c1bf34/init-config-reloader/0.log"
Jul 16 21:54:16.117306 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.117272    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"]
Jul 16 21:54:16.117679 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.117664    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b470be-e037-425a-98cc-df5796856850" containerName="revoke-keys"
Jul 16 21:54:16.117735 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.117681    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b470be-e037-425a-98cc-df5796856850" containerName="revoke-keys"
Jul 16 21:54:16.117735 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.117690    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:16.117735 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.117696    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:16.117833 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.117764    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fb0a43-a8ca-4838-931f-4e2919bab69e" containerName="maas-api"
Jul 16 21:54:16.120560 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.120542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.122743 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.122722    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8lb8\"/\"kube-root-ca.crt\""
Jul 16 21:54:16.122844 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.122817    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g8lb8\"/\"default-dockercfg-5rzmt\""
Jul 16 21:54:16.123516 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.123496    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8lb8\"/\"openshift-service-ca.crt\""
Jul 16 21:54:16.131543 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.131518    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"]
Jul 16 21:54:16.240657 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.240618    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-proc\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.240657 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.240658    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-lib-modules\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.240889 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.240767    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-sys\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.240889 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.240846    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xpkw\" (UniqueName: \"kubernetes.io/projected/90ecd94a-900b-4b57-a121-68053953d7c6-kube-api-access-9xpkw\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.240974 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.240889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-podres\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.341756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.341709    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-proc\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.341756 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.341758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-lib-modules\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.342026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.341805    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-sys\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.342026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.341846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xpkw\" (UniqueName: \"kubernetes.io/projected/90ecd94a-900b-4b57-a121-68053953d7c6-kube-api-access-9xpkw\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.342026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.341844    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-proc\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.342026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.341876    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-podres\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.342026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.341904    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-sys\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.342026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.341951    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-podres\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.342026 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.341968    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90ecd94a-900b-4b57-a121-68053953d7c6-lib-modules\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.350485 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.350458    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xpkw\" (UniqueName: \"kubernetes.io/projected/90ecd94a-900b-4b57-a121-68053953d7c6-kube-api-access-9xpkw\") pod \"perf-node-gather-daemonset-kk2q2\" (UID: \"90ecd94a-900b-4b57-a121-68053953d7c6\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.430731 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.430629    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:16.577426 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:16.577399    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"]
Jul 16 21:54:16.579728 ip-10-0-138-201 kubenswrapper[2582]: W0716 21:54:16.579698    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod90ecd94a_900b_4b57_a121_68053953d7c6.slice/crio-c550cbb4c7e47a886eb298352e78eda320a906b62b1a96c77a53e8049d57bb00 WatchSource:0}: Error finding container c550cbb4c7e47a886eb298352e78eda320a906b62b1a96c77a53e8049d57bb00: Status 404 returned error can't find the container with id c550cbb4c7e47a886eb298352e78eda320a906b62b1a96c77a53e8049d57bb00
Jul 16 21:54:17.097512 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:17.097475    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2" event={"ID":"90ecd94a-900b-4b57-a121-68053953d7c6","Type":"ContainerStarted","Data":"5cf9998c3e7e3d55a357519829df9747f34f95371f33f4bd30e453b909ebef5b"}
Jul 16 21:54:17.097727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:17.097525    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2" event={"ID":"90ecd94a-900b-4b57-a121-68053953d7c6","Type":"ContainerStarted","Data":"c550cbb4c7e47a886eb298352e78eda320a906b62b1a96c77a53e8049d57bb00"}
Jul 16 21:54:17.097727 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:17.097595    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:18.405662 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:18.405630    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-58cb7f5466-6jqf9_d0082008-2297-4548-9da2-51af0ceaeec4/console/0.log"
Jul 16 21:54:19.501493 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:19.501459    2582 scope.go:117] "RemoveContainer" containerID="25d83edc1456e11b0a951fed82c1f07365904928db597fdde4935641ec1ea6ef"
Jul 16 21:54:20.114367 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:20.114332    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" event={"ID":"2f0191b7-4baf-4d7b-9c0c-e54a286b927e","Type":"ContainerStarted","Data":"d5180c50c32c001e159a903145330ffd313d028bd62adccf9d6bb9bf678cc40e"}
Jul 16 21:54:20.134434 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:20.133978    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2" podStartSLOduration=4.13395534 podStartE2EDuration="4.13395534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:54:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:17.130228982 +0000 UTC m=+2321.187059894" watchObservedRunningTime="2026-07-16 21:54:20.13395534 +0000 UTC m=+2324.190786253"
Jul 16 21:54:20.361631 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:20.361540    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gwgdk_f791e45e-b730-4183-9b33-70be21cff13a/dns/0.log"
Jul 16 21:54:20.390117 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:20.390047    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gwgdk_f791e45e-b730-4183-9b33-70be21cff13a/kube-rbac-proxy/0.log"
Jul 16 21:54:20.508588 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:20.508557    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-772xx_e047fbb7-6b0e-4ad1-8348-de32e09d26ad/dns-node-resolver/0.log"
Jul 16 21:54:21.121303 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:21.121271    2582 generic.go:356] "Generic (PLEG): container finished" podID="2f0191b7-4baf-4d7b-9c0c-e54a286b927e" containerID="d5180c50c32c001e159a903145330ffd313d028bd62adccf9d6bb9bf678cc40e" exitCode=6
Jul 16 21:54:21.121303 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:21.121307    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" event={"ID":"2f0191b7-4baf-4d7b-9c0c-e54a286b927e","Type":"ContainerDied","Data":"d5180c50c32c001e159a903145330ffd313d028bd62adccf9d6bb9bf678cc40e"}
Jul 16 21:54:21.121600 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:21.121364    2582 scope.go:117] "RemoveContainer" containerID="25d83edc1456e11b0a951fed82c1f07365904928db597fdde4935641ec1ea6ef"
Jul 16 21:54:21.121658 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:21.121628    2582 scope.go:117] "RemoveContainer" containerID="d5180c50c32c001e159a903145330ffd313d028bd62adccf9d6bb9bf678cc40e"
Jul 16 21:54:21.121788 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:21.121771    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-85fbb945c6-fdbjg_dd9a270b-9a17-49ca-8ae0-11304f71184f/registry/0.log"
Jul 16 21:54:21.121902 ip-10-0-138-201 kubenswrapper[2582]: E0716 21:54:21.121881    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c_odh-ai-gateway-infra(2f0191b7-4baf-4d7b-9c0c-e54a286b927e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" podUID="2f0191b7-4baf-4d7b-9c0c-e54a286b927e"
Jul 16 21:54:21.144888 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:21.144860    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"]
Jul 16 21:54:21.153281 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:21.153257    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-64prw_063fca31-aa29-467d-b4f2-0a81b45d9707/node-ca/0.log"
Jul 16 21:54:22.270554 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:22.270521    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"
Jul 16 21:54:22.369394 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:22.369361    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-796f7d4c6-wjjfl_d2e98803-3f5b-4e4b-b04f-9adfd5045a74/kube-auth-proxy/0.log"
Jul 16 21:54:22.399449 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:22.399352    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0191b7-4baf-4d7b-9c0c-e54a286b927e-maas-api-service-ca\") pod \"2f0191b7-4baf-4d7b-9c0c-e54a286b927e\" (UID: \"2f0191b7-4baf-4d7b-9c0c-e54a286b927e\") "
Jul 16 21:54:22.399710 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:22.399686    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0191b7-4baf-4d7b-9c0c-e54a286b927e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f0191b7-4baf-4d7b-9c0c-e54a286b927e" (UID: "2f0191b7-4baf-4d7b-9c0c-e54a286b927e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:54:22.481759 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:22.481728    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gdmsm_596dec7c-a5fb-4aae-8f2d-5903673b4eb4/payload-pre-processing/0.log"
Jul 16 21:54:22.500650 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:22.500607    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0191b7-4baf-4d7b-9c0c-e54a286b927e-maas-api-service-ca\") on node \"ip-10-0-138-201.ec2.internal\" DevicePath \"\""
Jul 16 21:54:22.519949 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:22.519917    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-7zlxf_42863b88-d149-4131-8598-6921b305389b/payload-processing/0.log"
Jul 16 21:54:23.113790 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:23.113761    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-kk2q2"
Jul 16 21:54:23.133046 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:23.133021    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"
Jul 16 21:54:23.133046 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:23.133034    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c" event={"ID":"2f0191b7-4baf-4d7b-9c0c-e54a286b927e","Type":"ContainerDied","Data":"340d86fc313e69a451551ccdb081963a1cc3ed5b84f36ce1abdd23ef5e0b98fe"}
Jul 16 21:54:23.133308 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:23.133107    2582 scope.go:117] "RemoveContainer" containerID="d5180c50c32c001e159a903145330ffd313d028bd62adccf9d6bb9bf678cc40e"
Jul 16 21:54:23.156955 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:23.156918    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"]
Jul 16 21:54:23.162611 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:23.162581    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-62p5c"]
Jul 16 21:54:23.329229 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:23.329199    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m25k7_c9e92ac7-c0b5-4c0f-8249-b4140a74d1e4/serve-healthcheck-canary/0.log"
Jul 16 21:54:24.168667 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:24.168638    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pklzw_9ca933aa-7524-4ca8-994b-b0a96e693952/kube-rbac-proxy/0.log"
Jul 16 21:54:24.195545 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:24.195514    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pklzw_9ca933aa-7524-4ca8-994b-b0a96e693952/exporter/0.log"
Jul 16 21:54:24.225699 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:24.225669    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pklzw_9ca933aa-7524-4ca8-994b-b0a96e693952/extractor/0.log"
Jul 16 21:54:24.506495 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:24.506406    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f0191b7-4baf-4d7b-9c0c-e54a286b927e" path="/var/lib/kubelet/pods/2f0191b7-4baf-4d7b-9c0c-e54a286b927e/volumes"
Jul 16 21:54:27.354358 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:27.354326    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2_255465d7-fbd4-4536-baf0-2a1b696f322f/manager/0.log"
Jul 16 21:54:27.360490 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:27.360461    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-796f7d4c6-wjjfl_d2e98803-3f5b-4e4b-b04f-9adfd5045a74/kube-auth-proxy/0.log"
Jul 16 21:54:27.386000 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:27.385971    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4b6cd7d5-6v5d2_255465d7-fbd4-4536-baf0-2a1b696f322f/copy-manifests/0.log"
Jul 16 21:54:27.449582 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:27.449548    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gdmsm_596dec7c-a5fb-4aae-8f2d-5903673b4eb4/payload-pre-processing/0.log"
Jul 16 21:54:27.471798 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:27.471704    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-7zlxf_42863b88-d149-4131-8598-6921b305389b/payload-processing/0.log"
Jul 16 21:54:28.745064 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:28.745034    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-796f7d4c6-wjjfl_d2e98803-3f5b-4e4b-b04f-9adfd5045a74/kube-auth-proxy/0.log"
Jul 16 21:54:28.821654 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:28.821608    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gdmsm_596dec7c-a5fb-4aae-8f2d-5903673b4eb4/payload-pre-processing/0.log"
Jul 16 21:54:28.842867 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:28.842832    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-7zlxf_42863b88-d149-4131-8598-6921b305389b/payload-processing/0.log"
Jul 16 21:54:28.895527 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:28.895502    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78cbff6c4-kcpzw_707c0999-19c7-45b6-8e20-6076dad3bb50/manager/0.log"
Jul 16 21:54:28.953424 ip-10-0-138-201 kubenswrapper[2582]: I0716 21:54:28.953396    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rz7d4_c4fad4a4-41a4-46de-a2eb-0ba9783a8b89/openshift-lws-operator/0.log"